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ABSTRACT 


In  2009,  the  Naval  Oceanographic  Office  was  tasked  with  developing  a  product  that  uses 
forecasted  meteorological  conditions  and  historical  pirate  incidents  to  predict  locations 
conducive  to  pirate  activity  in  the  Somali  Basin  Region  and  the  Gulf  of  Aden.  This 
resulted  in  the  development  of  the  Piracy  Performance  Surface  (PPS)  model,  whose 
outputs  are  briefed  daily  to  the  Commander  of  the  United  States  Naval  Forces  Central 
Command  and  Combined  Maritime  Forces  in  Bahrain.  The  Next-generation  PPS  (PPSN) 
model  uses  simulation  to  provide  as  output,  a  forecast  of  relative  pirate  presence 
probability  over  time.  Effective  March  1,  2011,  the  name  of  PPSN  has  been  changed  to 
the  Pirate  Attack  Risk  Surface  (PARS)  model. 

This  research  includes  interviews  with  counter-piracy  forces  that  led  to 
recommended  changes  in  the  PPSN  model.  In  addition,  using  robust  and  realistic 
experimental  designs,  this  research  identifies  the  significant  intelligence  factors  of  the 
PPSN  model.  This  gathered  information  is  being  used  to  refine  these  input  variables  to 
achieve  maximum  performance  of  the  PPSN  model.  This  research  also  unveiled  input 
variables  that  are  influential  in  the  computing  memory  requirements  and  program 
runtime.  This  information  is  being  used  to  focus  efforts  on  setting  these  variables  to 
realistic  levels  without  sacrificing  the  model's  efficiency  and  effectiveness.  Finally,  the 
results  of  this  thesis  allow  for  quick  turnaround  of  updates  to  the  PPSN  model  in  response 
to  gathered  intelligence. 
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EXECUTIVE  SUMMARY 


The  instability  and  lack  of  government  in  Somalia  have  caused  a  rapid  increase  in  pirate 
activity  within  the  Somali  Basin  Region  (SBR)  and  the  Gulf  of  Aden  (GOA).  Due  to  this 
increase,  maritime  security  and  stability  in  these  regions  are  severely  threatened.  Somali 
pirate  attacks  have  taken  the  lives  of  innocent  people,  held  sea-going  persons  hostage, 
and  cost  the  world's  economy  billions  of  dollars  annually  (Bowden,  Hurlburt,  Aloyo, 
Marts,  &  Lee,  2010,  pp.  12).  For  this  reason,  President  Barack  Obama  initiated  an 
executive  order  on  April  12,  2010  to  deal  with  the  threat  of  piracy  (Executive  Order  No. 
13,536,  2010).  President  Obama  stated  that  the  collapse  of  security  and  the  continuation 
of  violence  in  the  Somali  region  in  the  form  of  piracy  and  anned  robbery  on  the  high  seas 
are  a  threat  to  the  United  States’  (U.S.)  national  security  and  foreign  policy  of  the  United 
States  and  must  be  dealt  with  accordingly. 

U.S.  Naval  Forces  Central  Command  (NAVCENT)  Commander  Vice  Admiral 
Mark  Fox  and  U.S.  Sixth  Fleet  Commander  Vice  Admiral  Harry  Harris,  Jr.  have  been 
tasked  with  finding  a  solution  to  deter,  control,  and  ultimately  eliminate  pirating  within 
the  GOA  and  SBR  within  the  U.S.  Fifth  and  Sixth  Fleets’  operating  waters  by  returning 
safety,  stability,  and  peace  to  these  regions.  Bahrain,  being  strategically  located  within 
the  Middle  East,  has  been  established  as  the  U.S.  Fifth  Fleet  and  Combined  Maritime 
Forces  (CMF)  Headquarters  for  the  Combined  Maritime  Forces,  the  North  Atlantic 
Treaty  Organization  (NATO),  and  the  European  Union  (EU);  meanwhile,  the  U.S.  Sixth 
Fleet  headquarters  resides  in  Naples,  Italy. 

To  combat  the  on-going  concerns  with  piracy,  Naval  Oceanographic  Office 
(NAVO)  researchers  were  tasked  by  Rear  Admiral  Titley,  Oceanographer  and  Navigator 
of  the  Navy,  to  assist  in  the  fight  against  piracy.  This  tasking  occurred  just  days  after  the 
MV Maersk  Alabama,  a  U.  S. -flagged  ship,  was  hijacked  by  Somali  pirates  in  the  Indian 
Ocean  on  April  18,  2009.  The  Piracy  Performance  Surface  (PPS)  model  was  completed 
approximately  two  weeks  after  this  incident,  and  has  undergone  a  few  revisions  since  its 
development.  The  success  of  the  PPS  model  led  to  the  Naval  Meteorology  and 

Oceanography  Command  leadership  to  devote  efforts  to  pursue  a  more  advanced  anti- 

xvii 


piracy  model,  the  Next-generation  PPS  (PPSN).  On  March  1,  2011,  the  PPSN  model's 
name  changed  to  the  Pirate  Attack  Risk  Surface  (PARS).  Dr.  James  Hansen  of  the  Naval 
Research  Laboratory  (NRL)  in  Monterey,  California,  was  tasked  with  developing  this 
next  generation  probabilistic  forecasting  tool. 

The  PPSN  model  is  a  Monte  Carlo  probabilistic  forecasting  tool  that  takes  as 
input  environmental  conditions,  pirate  concept  of  operations,  and  gathered  intelligence  to 
produce  as  output  the  forecast  of  pirate  presence  probability  as  a  function  of  latitude, 
longitude,  and  time.  The  model  simulates  pirate  action  groups  (PAGs),  where  a  PAG  is  a 
group  of  pirate  dhows  that  are  attached  to  a  mother  dhow  (a  larger  vessel)  that  is  typically 
used  as  a  logistical  hub  for  extending  distances  from  land  and  mission  duration.  The 
PPSN  outputs  a  forecast  of  relative  pirate  presence  probability  over  a  72-hour  forecast 
time  period  in  0.2-hour  time  steps  with  a  probability  surface  generated  at  12-hour  time 
steps  (See  Figure  1).  As  of  the  date  of  this  thesis,  the  PPSN  model  is  the  only  known 
Navy  product  that  couples  METOC  and  INTEL. 


kxifptude  (degrees  east)  longitude  (degrees  east) 

Figure  1.  The  PPSN  model  output  displaying  the  forecast  of  relative  pirate  presence 
probability  with  the  24-hour  forecast  (left)  and  the  72-hour  forecast  (right).  Red 
corresponds  to  the  highest  probability  forecast  of  pirate  location. 

This  thesis  research  includes  visits  and  interviews  with  users  of  PPS  and  other 
counter-piracy  forces  to  understand  the  operators'  needs  and  to  gain  intelligence  on  the 
most  current  pirate  CONOPS  within  the  SBR  and  GOA.  This  gathered  intelligence  led  to 
recommended  changes  in  the  PPSN  code  to  include  the  addition  of  PAG  search  patterns, 
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the  implementation  of  PAG  operating  boxes  and  their  mapping,  and  the  understanding  of 
pirate  terminology  used  by  military  forces  combating  piracy. 

Key  findings  this  research  addresses  are:  (1)  which  input  parameters  can  be 
adjusted  to  decrease  run  time  and  computing  memory  requirements,  (2)  which  input 
parameters  interact  most  strongly  with  METOC,  (3)  which  input  parameters  are  the  most 
influential  to  the  PPSN  output,  and  (4)  what  improvements  can  be  made  to  enhance  the 
PPSN  model. 

Three  key  input  parameters  that  impact  the  computer  memory  requirement,  which 
are  also  highly  correlated  with  runtime,  are  the  PAG's  mission  length,  the  number  of 
PAG  replications  in  the  simulation,  and  the  number  of  P AGs.  The  analysis  by  the  author 
concludes  that  the  values  inputted  for  each  of  these  variables,  in  particular,  the  PAG's 
mission  length,  should  be  considered  with  the  utmost  care  as  their  associated  values 
greatly  impact  the  forecast  of  relative  pirate  presence  probability. 

The  search  pattern  the  PAGs  use  when  attempting  to  locate  a  target  vessel  of 
opportunity  is  the  most  significant  factor  in  the  PPSN  model.  Operators  and  military 
members  combating  piracy  should  attempt  to  acquire  intelligence  on  the  specific  search 
pattern  that  PAGs  use  in  the  SBR  and  GOA. 

The  probability  that  PAGs  have  meteorology  and  oceanography  information  prior 
to  leaving  their  base  location  is  highly  influential  and  interacts  with  the  most  important 
variable,  the  PAG's  search  pattern. 

In  addition,  using  systematic  and  robust  experimental  designs,  this  research  made 
PPSN  more  robust  and  realistic  by  breaking  the  code  in  different  ways,  resulting  in 
modifications  by  the  developer  to  make  PPSN  better  and  more  robust. 
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I.  INTRODUCTION 


A.  BACKGROUND  AND  MOTIVATION 

The  instability  and  lack  of  government  in  Somalia  have  caused  a  rapid  increase  in 
pirate  activity  within  the  Somali  Basin  Region  (SBR)  and  the  Gulf  of  Aden  (GOA).  The 
SBR  is  the  region  that  lies  directly  between  Somalia  and  India  in  the  Indian  Ocean.  The 
GOA  is  the  body  of  water  located  between  the  Horn  of  Africa  (HO A)  and  Yemen.  Piracy 
in  these  regions  has  not  only  threatened  the  stability  and  safety  of  ocean-going  vessels 
and  their  crew  members,  but  has  also  cost  the  world's  economy  over  billions  of  dollars 
(Bowden,  Hurlburt,  Aloyo,  Marts,  &  Lee,  2010,  pp.  12). 1  For  this  reason,  President 
Barack  Obama  initiated  Executive  Order  13536  on  April  12,  2010  to  deal  with  the  threat 
of  piracy.  President  Obama  stated  that  the  collapse  of  security  and  the  continuation  of 
violence  in  the  Somali  region  in  the  form  of  piracy  and  armed  robbery  on  the  high  seas  is 
a  threat  to  the  United  States’  national  security  and  foreign  policy  of  the  United  States  and 
must  be  dealt  with  accordingly. 

Two  U.S.  Combatant  Commands  (COCOMs)  are  tasked  with  solving  the  Somalia 
piracy  problem,  U.S.  Africa  Command  (AFRICOM)  under  General  Carter  Ham  (From 
http://www.africom.mil),  U.S.  Army,  and  Central  Command  under  General  James  Maths 
(From  http://www.centcom.mil),  U.S.  Marine  Corps  (USMC).  Both  face  an  extremely 
complex  situation  with  combating  piracy  that  originates  in  the  four  distinct  and 
independent  political  regions  that  make  up  Somalia  (Figure  2),  and  finishes  with 
hijacking  vessels  at  sea.  AFRICOM  is  currently  conducting  the  African  Partnership 
Station  (APS)  2011  with  22  African-partnered  nations  (LT  S.  Hall,  personal 
communication,  January  31,  201 1).2  CENTCOM  is  responsible  for  the  waters 
surrounding  the  SBR  and  GOA,  and  is  conducting  Maritime  Security  Operations  (MSO) 


1  The  costs  associated  with  piracy  include  ransom  costs,  insurance  increases,  re-directing  commercial 
ships,  additional  security,  the  use  of  military  vessels,  legal  jurisdiction  for  captured  pirates,  and 
organizations  that  help  combat  piracy. 

2  The  author  was  forwarded  this  information  through  LT  Stacey  Hall.  AFRICOM  sent  LT  B.  Le  to 
present  the  African  Partnership  Station  (APS)  2011  on  behalf  of  AFRICOM  (Africa  Combatant  Command) 
on  February  2,  2011. 
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with  Combined  Task  Force  (CTF)  150  and  CTF  151.  As  the  reader  can  see,  complexity 
exists  in  the  Somali  region  and  special  attention  must  be  exhibited  by  the  Commanders  of 
the  two  COCOMs  with  a  partnership  for  countering  piracy  operations  on  land  and  at  sea. 
An  additional  layer  of  complexity  exists  with  a  separate  U.S.  fleet  reporting  to  each 
COCOM  and  both  fleets  sharing  the  responsibility  for  the  safety  and  stability  of  the 
waters  surrounding  Africa. 


Figure  2.  Map  of  the  four  political  regions  of  Somalia  used  by  the  Combined  Maritime 
Forces  assessment  team.  Region  one  is  defined  as  Somaliland;  region  two  is 
defined  as  Puntland;  region  three  is  defined  as  Galmudug;  and  region  four  is 
Harakat  al-Shabaab  Mujahideen  (From  Wikipedia,  2011). 


U.S.  Naval  Forces  Central  Command  (NAVCENT);  Commander,  U.S.  Fifth  Fleet 
and  Commander,  Combined  Maritime  Forces  (CMF),  Vice  Admiral  Mark  Fox;3  and 


3  For  detailed  information  on  Admiral  Fox  and  the  U.S.  Fifth  Fleet's  mission  statement  and  recent 
operations  visit  http://www.cusnc.navy.mil. 
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Commander,  U.S.  Sixth  Fleet,  Vice  Admiral  Harry  Harris,  Jr.,4  have  been  tasked  with 
finding  a  solution  to  deter,  control,  and  ultimately  eliminate  pirating  within  the  GOA  and 
SBR  within  the  U.S.  Fifth  and  Sixth  Fleets’  operating  waters  by  returning  safety, 
stability,  and  peace  to  approximately  two  million  square  miles  of  ocean  water 
surrounding  Somalia.5  Bahrain,  being  strategically  located  within  the  Middle  East,  has 
been  established  as  the  U.S.  Fifth  Fleet  and  Combined  Maritime  Forces  (CMF) 
headquarters  for  the  Big  Three:  Combined  Maritime  Forces  (comprised  of  25  nations), 
the  North  Atlantic  Treaty  Organization  (NATO),  and  the  European  Union  (EU); 
meanwhile,  the  U.S.  Sixth  Fleet  headquarters  resides  in  Naples,  Italy.  LCDR  K.  Lutz6 
(personal  communication,  June  8,  2010)  stated  that  other  nations,  such  as  China  and 
India,  are  also  working  to  defeat  these  opportunistic  ocean-driven  thieves,  but  these  two 
nations  conduct  missions  not  in  affiliation  with  the  U.S.,  NATO,  or  the  EU. 

In  the  2010  National  Security  Strategy  (NSS),  President  Obama  stated  that  with 
the  continued  threat  of  unstable  nations  the  U.S.  military  will  constantly  be  called  to 
provide  military  security  and  stability  as  part  of  globalization  (President  of  the  United 
States,  2010,  pp.  12).  Follow-on  to  the  NSS  is  the  2011  National  Military  Strategy 
(NMS)  guidance.  In  this  paper,  Admiral  Mike  Mullen,  Chairman  to  the  Joint  Chiefs  of 
Staff,  stated  that  the  Joint  Forces  will  continue  to  address  and  combat  the  violent 
extremists  that  have  sought  a  safe  haven  in  the  SBR  and  GOA  regions  as  these  extremists 
(pirates)  remain  a  transnational  security  threat  (CJCS,  2011).  Senior  military  leaders  are 
in  a  difficult  position  and  General  Mattis  is  a  prime  example.  As  CENTCOM 
Commander,  General  Mattis  must  divide  his  military  assets  (air,  land,  and  sea)  between 
the  instability  in  the  Middle  East  and  the  “rash  of  piracy”  in  the  GOA  and  SBR  (LCDR 
K.  Lutz,  personal  communication,  June  7,  2010).  How  to  properly  allocate  military 
forces  in  order  to  sustain  and  ultimately  eliminate  piracy  is  a  question  that  leading  U.S. 


4  For  detailed  information  on  Admiral  Harris  and  U.S.  Sixth  Fleet's  mission  statement  and  recent 
operations  visit  http://www.naveur-navaf.navy.mil. 

5  For  detailed  information  on  Central  Command  (CENTCOM)  visit  http://www.centcom.mil. 

6  LCDR  K.  Lutz,  UK,  held  the  intelligence  officer  (INTEL  officer)  position  for  the  CMF  assessment 
team  in  Bahrain  during  the  author’s  operations  research  experience  tour. 
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military  and  civilian  officials  are  looking  to  answer.  This  thesis  seeks  to  support  the 
mission  by  improving  the  tools  available  to  decision  makers  to  allocate  forces, 
specifically  forces  combating  piracy. 

The  first  operational  piracy  prediction  model,  Piracy  Performance  Surface  (PPS), 
was  developed  in  2009  by  the  Naval  Oceanographic  Office  (NAVO)  at  Stennis  Space 
Center.  The  successes  of  PPS  led  to  follow-on  research  for  the  development  of  a  pirate 
probabilistic  tool,  the  Next-generation  PPS  model  (PPSN).7  This  tool  produces  as  output 
a  forecast  of  relative  pirate  presence  probability  as  a  function  of  latitude  and  longitude 
over  time.  This  thesis  was  conducted  parallel  to  the  development  of  the  PPSN  by  the 
Naval  Research  Laboratory  (NRL)  Monterey  and  NAVO  and  explores  the  operational 
usefulness,  product  stability,  and  output  consistency  of  PPSN.  In  addition,  this  research 
includes  visits  and  interviews  with  users  of  the  PPS  model  and  other  counter-piracy 
forces  to  understand  their  needs  and  pirate  tactics,  which  led  to  recommended  changes  in 
PPSN. 

B.  RESEARCH  QUESTIONS 

This  thesis  addresses  the  following  questions: 

1.  What  improvements  can  be  made  to  the  operational  version  of  the  PPSN 
model  that  will  enhance  the  product’s  output? 

2.  Which  PPSN  input  parameters  can  be  adjusted  to  decrease  computing 
memory  requirements  and  run  time? 

3.  Which  PPSN  input  parameters  interact  most  strongly  with  Meteorology  and 
Oceanography  (METOC)  conditions? 

4.  Which  PPSN  input  parameters  are  the  most  influential  in  detennining  the 
distribution  of  relative  risk  of  pirate  presence  in  the  PPSN? 


7  For  continuity  in  this  thesis,  the  name  of  the  next-generation  Piracy  Performance  Surface  model 
(PPSN)  will  be  used.  Effective  March  1,  201 1  the  name  of  the  PPSN  model  was  changed  to  the  Pirate 
Attack  Risk  Surface  model,  PARS. 
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C.  BENEFITS  OF  THE  STUDY 

Using  carefully  designed  simulation  experiments,  this  research  identifies  the  most 
important  factors  driving  the  PPSN  output.  This  information  can  be  used  for  refining 
intelligence  inputs  and  the  PPSN  model  development  efforts  to  implement  the  most 
influential  functionality.  These  results  can  also  be  used  to  optimize  parameter  selection  to 
allow  quick  turnaround  of  updates  to  the  product  in  response  to  INTEL,  within 
computational  constraints,  and  without  sacrificing  product  quality.  In  addition,  the 
information  gathered  in  this  thesis  will  be  used  to  assist  the  military  INTEL  community 
(primarily  in  affiliation  with  the  Big  Three  in  Bahrain)  with  PPSN. 

D.  THESIS  ORGANIZATION 

Chapter  II  provides  the  reader  with  a  literature  brief  that  describes  the  basis  for 
this  thesis  research.  Chapter  III  describes  the  evolution  of  the  efforts  of  the  U.S.  in  the 
development  of  piracy  models,  and  in  particular  the  development  of  PPSN.  Chapter  IV 
defines  the  choice  of  independent  variables  for  simulation  modeling  and  the  output 
metrics  used  to  evaluate  the  forecast  of  relative  pirate  presence  probability.  Chapter  V 
discusses  each  simulation  experimental  design.  Chapter  VI  examines  the  results  of  the 
experiments,  including  multivariate  correlation,  stepwise  regression,  sensitivity  analysis, 
and  classification  and  regression  trees.  Chapter  VII  draws  conclusions  and  makes  follow- 
on  recommendations. 
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II.  LITERATURE  REVIEW 


In  this  chapter,  piracy  will  be  defined  and  the  following  topics  covered:  costs  and 
challenges  created  by  piracy,  current  piracy  concept  of  operations,  other  nations'  potential 
piracy  tools,  and  the  development  of  experimental  designs. 

A.  DEFINITIONS  OF  PIRACY 

Piracy  has  plagued  the  world  for  thousands  of  years.  However,  the  U.S. 
government  did  not  clearly  define  piracy  for  many  years.  In  fact,  the  U.S.  Congress  did 
not  take  an  aggressive  role  in  defining  piracy  for  the  U.S.  judicial  system  until  December 
2010  (Mason,  2010).  When  it  finally  did,  the  definition  took  its  origins  from  the  1982 
United  Nations  Convention  on  the  Law  of  the  Sea  (UNCLOS)  Article  101  that  defines 
piracy  as: 

(a)  Any  illegal  acts  of  violence  or  detention,  or  any  act  of  depredation, 
committed  for  private  ends  by  the  crew  or  passengers  of  a  private  ship  or  a 
private  aircraft  and  directed: 

(i)  On  the  high  seas,  against  another  ship  or  aircraft,  or  against 
persons  or  property  on  board  such  ship  or  aircraft; 

(ii)  Against  a  ship,  aircraft,  persons  or  property  in  a  place  outside 
the  jurisdiction  of  any  State; 

(b)  Any  act  of  voluntary  participation  in  the  operation  of  a  ship  or  of  an 
aircraft  with  knowledge  of  facts  making  it  a  pirate  ship  or  aircraft; 

(c)  Any  act  of  inciting  or  of  intentionally  facilitating  an  act  described  in 
subparagraph  (a)  or  (b). 

Other  variations  of  the  definition  of  piracy  include  the  following:  “an  act  of 
violence,  or  the  threat  of  violence,  against  a  ship  in  international  waters  (outside  a  state’s 
geographic  sovereignty)  with  the  intent  to  commit  crimes  such  as  theft,  hijacking, 
murder,  and  kidnapping”  (Kline,  2010,  pp.  69);  and  “an  act  of  boarding  or  attempting  to 
board  any  ship  with  the  apparent  intent  to  commit  theft  or  any  other  crime  and  with  the 
apparent  intent  or  capability  to  use  force  in  furtherance  of  that  act”  (Chalk,  2008,  pp.  3). 
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Many  variations  of  the  definition  of  piracy  exist,  but  the  key  to  combating  piracy 
begins  by  identifying  the  root  cause  of  this  multi-national  threat  (Mason,  2010).  Two 
questions  have  come  into  the  political  arena  (Goodman,  2009): 

(a)  Did  piracy  develop  because  of  illegal  fishing  in  the  economic 

exclusion  zone  (EEZ)  of  the  SBR? 

Or 

(b)  Did  piracy  develop  from  an  unstable  Somali  government? 

The  former  CMF  legal  officer,  CDR  David  Teasdale,  United  Kingdom  (UK), 
stated  that  the  UNCLOS  has  clearly  defined  piracy,  but  the  real  debate  surrounding 
piracy  is  a  “policy  debate  rather  than  a  legal  one"  (personal  communication,  July  10, 
2010).  The  policy  debate  exists  between  the  U.S.,  NATO,  and  the  European  Union.  The 
UK  Ambassador  at  the  Security  Council  Debate  on  Somalia  Piracy  stated,  "The  United 
Kingdom  believes  that  the  best  prospect  of  sustainable  results  will  come  from  further 
enhancing  the  international  community’s  efforts  to  build  the  capacity  of  regional  states  to 
prosecute  and  imprison  those  responsible  for  acts  of  piracy"  (Grant,  2010,  para.  7).  Grant 
(2010)  later  stated  that  he  believes  the  root  cause  of  piracy  is  a  direct  result  of  the 
instability  of  the  Somali  government. 

B.  COSTS  AND  CHALLENGES  CREATED  BY  PIRACY 

Chalk  (2008)  stated  that  the  International  Maritime  Bureau  (IMB)  "estimates  that 
piracy  costs  the  shipping  industry  anywhere  from  $1  billion  to  $16  billion  a  year”  (pp. 
16).  Paying  the  ransom  money  is  more  cost-effective  than  implementing  protective 
measurers  onboard,  but  this  method  is  a  quick  fix  to  the  ultimate  problem.  A  Cooperative 
Strategy  for  the  21st  Century  Seapower  recognizes  the  need  for  international  partnership 
(Roughead,  Conway,  &  Allen,  2007).  “This  strategy  stresses  an  approach  that  integrates 
seapower  with  other  elements  of  national  power,  as  well  as  those  of  our  friends  and 
allies”  (Roughead  et  ah,  2007,  para.  1).  The  commercial  shipping  community  as  well 
must  continue  to  practice  maritime  awareness  as  modern  day  pirates  are  better  trained 
and  equipped  with  semi-automatic  machine  guns,  varieties  of  rocket  propelled  grenades, 
and  small  arms  (Best  Management  Practices  [BMP3],  2010). 
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C.  CURRENT  PIRACY  CONCEPT  OF  OPERATIONS 

Pirate  concept  of  operations  (CONOPS),  also  known  as  pirate  tactics,  has  been 
changing  and  evolving  as  the  number  of  successful  paid  ransoms  has  increased.  In  April 

2010,  Rear  Admiral  Peter  Hudson,  British  EU  Admiral,  addressed  a  new  pirate  CONOPS 
swarming  tactic  that  allowed  pirate  action  groups  to  supersaturate  the  area  once  a  target 
of  opportunity  had  been  identified  (Childs,  2010).  Pirates  have  also  extended  their 
operating  area  to  1,000  nautical  miles  from  the  shoreline  because  commercial  vessels  are 
no  longer  hugging  the  Kenya  and  Somali  coastline  (Figure  3).  Admiral  Hudson  was  also 
quoted  in  saying  (Childs,  2010): 

Five  years  ago,  the  maximum  range  of  attacks  was  287  km  (165  nautical 
miles).  Recently  a  ship  was  hijacked  2,037  km  (1,100  nautical  miles) 
from  the  Somali  coast  -  and  only  926  km  (500  miles)  from  the  coast  of 
India. 

Weather  conditions  also  play  a  significant  role  in  impacting  pirate  CONOPS. 
Historical  METOC  and  INTEL  reports  have  shown  that  pirate  activity  decreases  in  the 
SBR  during  the  biannual  monsoon  season.8  This  decrease  is  due  to  the  heightened  wind 
and  wave  conditions  that  make  the  sea  states  unfavorable  for  pirating  in  the  SBR.  In 

2011,  though,  new  pirate  CONOPS  have  enabled  the  pirate  action  groups  (PAGs)  to 
operate  in  these  adverse  weather  conditions.  Admiral  Fox  stated  (Fellman,  2011): 

The  monsoon  season  once  kept  pirate  skiffs  ashore,  but  mother  ships  allow 
for  year-round  piracy.  About  eight  of  these  “pirate  action  groups”  are 
spread  throughout  the  region,  he  said. 


8  See  the  Piracy  Analysis  and  Warning  Weekly  (PAWW)  Report  (2010)  for  an  example  of  this  report. 
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Figure  3.  This  picture  shows  the  extension  of  piracy  missions  from  2005  through  2011. 
In  2005,  pirates  were  seen  operating  165  nautical  miles  in  the  SBR  and  GOA,  but 
in  2011,  pirates  are  operating  1,300  nautical  miles  from  the  coastline  in  these 
areas  (Somalia's  states,  regions,  and  districts)  (From  Childs,  2010). 


PAG  is  the  tenninology  used  by  the  Big  Three  to  refer  to  a  mother  dhow  and  its 
adjoining  pirate  dhows.  Typically,  a  mother  dhow  is  a  larger  vessel,  possibly  a  Boston 
whaler,  used  as  the  logistical  hub  to  extend  the  longevity  in  time  and  distance  of  the 
piracy  mission.  Pirate  dhows  are  often  attached  to  the  mother  dhow  with  the  boarding 
ladder,  which  is  an  aluminum  ladder  (Figure  4)  (Agence  France  Presse  [AFP],  2009). 
When  pirates  have  located  a  vessel  of  opportunity,  the  pirates  will  use  marine  equipment 
such  as  grappling  hooks  to  board  the  vessel.  Pirates  onboard  can  be  equipped  with 
satellite  phones,  global  positioning  system  (GPS),  rocket-propelled  grenade  launchers 
with  warheads  (RPGs),  and  AK-47s  (Floureld,  2010).  Commercial  vessels  traveling  at 
slow  speeds  and  having  low  freeboards  are  typically  easier  targets  of  opportunity. 
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Figure  4.  Picture  on  the  left  is  a  mother  dhow  for  logistical  support  at  sea  and  three 
pirate  dhows  attached  for  the  attacking  of  a  commercial  vessel  (From  Operation 
Ocean  Shield,  2010).  The  picture  on  the  right  is  a  pirate  dhow  and  pirates  (From 
Ocean  Shield,  2010). 

In  the  past,  pirates  have  not  been  known  to  kill  the  crew  of  the  pirated  vessel.  On 
February  22,  2011,  pirates  captured  and  killed  four  Americans  onboard  the  Quest,  a  58- 
foot  yacht  (Wadhams,  2011).  These  killings  sparked  questions  about  the  effectiveness  of 
the  military  in  deterring  piracy.  Hillary  Clinton,  U.S.  Secretary  of  State,  stated,  "We've 
got  to  have  a  more  effective  approach  to  maintaining  security  on  the  seas,  in  the  ocean 
lanes  that  are  so  essential  to  commerce  and  travel"  (Wadhams,  2011). 

D.  OTHER  POTENTIAL  PIRACY  TOOLS 
1.  United  Kingdom  Global  Model 

As  part  of  the  Big  Three,  the  UK  has  its  own  forecasting  model,  the  UK  Global 
Model  (UKGM),  for  "ease  of  maneuver  and  to  optimize  precision  weapons"  (Ritchie, 
2010).  This  model  is  run  four  times  per  day  and  provides  as  output  a  forecast  for  the 
following  six  days.  Wave  Watch  III  provides  the  wave  characteristics  and  the 
Forecasting  Ocean  Assimilation  Model-Nucleus  of  European  Modeling  of  the  Ocean 
(FOAM -NEMO)  provides  environmental  information  such  as  salinity  and  sea 
temperature  for  the  model.  The  UKGM,  with  inputs  from  other  systems,  can  be  used  to 
provide  advice  for  likely  enemy  courses  of  action  (COAs),  but  this  model  does  not 
incorporate  the  INTEL  about  PAG  CONOPS. 
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2.  Technical  Cooperation  Program  Maritime  Group 

In  October  2008,  Australia,  Canada,  New  Zealand,  the  UK,  and  the  U.S.  agreed  to 
develop  a  piracy-based  toolset  that  would  support  the  efforts  to  counter  the  increasing 
threat  of  piracy  with  specific  attention  paid  to  the  HOA.  This  toolset  is  the  Technical 
Cooperation  Program  Maritime  Group  (TTCP  MAR).  Discussions  held  during  a  TTCP 
MAR  meeting  in  September  2009  in  Halifax,  Canada,  addressed  the  desire  to  "conduct  a 
Red  Teaming  Exercise  to  help  predict  how  pirate  tactics  and  strategies  may  adapt  and 
evolve  as  their  current  operations  are  impacted  by  coalition  interdiction  efforts"  (D. 
DaSilva,  personal  communication,  June  11,  20 1 0).9  The  developers  of  the  TTCP  MAR 
were  interested  in  learning  about  the  author's  research  and  how  it  could  enhance  TTCP 
MAR’s  development. 

E.  EXPERIMENTAL  DESIGNS 

1.  Design  of  Experiments 

Design  of  experiments  (DOE)  has  an  extensive  background  and  a  variety  of 
applications  in  the  operations  research  and  simulation  community.  Sanchez  (2005) 
describes  DOE  as  the  ability  of  "the  analysts  to  examine  many  more  factors  than  would 
otherwise  be  possible,  while  providing  insights  that  could  not  be  gleaned  from  trial-and- 
error  approaches  or  by  sampling  factors  one  at  a  time"  (pp.  69).  DOE  can  be  thought  of 
as  an  enabler  that  when  properly  implemented  helps  to  reduce  correlation  among  input 
variables  while  providing  a  robust  design  that  eliminates  duplication  in  the  data  analysis, 
which  leads  to  greater  statistical  power. 

As  described  by  Kleijnen,  Sanchez,  Lucas,  and  Cioppa  (2005),  DOE  provides 
effective  and  efficient  results  without  using  a  "trial-by-error  approach"  (pp.  263). 
Kleijnen  et  al.  (2005)  defines  basic  objectives  analysts  should  have  in  developing  their 


9  yjQp  mar  information  was  provided  by  D.  DaSilva  (UK  civilian  contractor  for  the  CMF 
assessment  team)  to  the  author  as  a  result  of  the  author's  operations  research  experience  tour  to  the  CMF  in 
Bahrain  on  June  11,2010. 
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DOE  prior  to  formulating  the  meta-model.  Kleijnen  et  al.  (2005)  recommends  that  the 
DOE  must  be  an  integral  part  in  the  developmental  stages  of  multivariate  polynomial 
modeling  and  should  continue  thereafter: 

DOE  can  uncover  detailed  insight  into  the  model’s  behavior,  cause  the 
modeling  team  to  discuss  in  detail  the  implications  of  various  model 
assumptions,  help  frame  questions  when  the  analysts  may  not  know  ahead 
of  time  what  questions  should  be  asked,  challenge  or  confirm  expectations 
about  the  direction  and  relative  important  of  factor  effects,  and  even 
uncover  problems  in  the  program  logic,  (pp.  266) 

When  analyzing  the  output,  analysts  should  be  aware  of  nonlinear  effects, 
discontinuous  effects,  chaotic  behavior,  and  a  simulation  spin-up  period  (Kleijnen  et  al., 
2005).  These  model  effects  can  lead  to  the  wrong  analytical  result. 

2.  Nearly  Orthogonal  Latin  Hypercubes 

Depending  on  the  question(s)  to  be  answered,  the  modeler  may  be  focused  on 
orthogonality  of  input  factors.  Orthogonality  means  the  model's  factors  are 
uncorrelated — the  cross  product  of  any  two  input  vectors  in  the  model’s  space  is  equal  to 
zero.  If  there  is  multicolinearity  among  input  factors,  then  the  differences  in  standard 
error  compared  to  the  best-fit  model  may  be  extremely  large  and  lead  to  inconclusive 
analytical  results  (Kleijnen  et  al.,  2005). 

To  provide  a  good  DOE  for  many  applications,  Cioppa  and  Lucas  (2007)  develop 
an  algorithm  to  provide  nearly  orthogonal  Latin  hypercube  (NOLH)  experimental 
designs.  Cioppa  and  Lucas  start  with  the  goal  of  estimating  simple  yet  common  first- 
order  effects  in  a  model.  In  general,  to  estimate  such  a  model,  the  number  of  simulation 
runs  increases  significantly  with  the  number  of  factors,  which  may  be  prohibitive.  Cioppa 
and  Lucas  define  two  objectives,  pmax  and  the  condition  number,  cond  ( XT  X ),  that 
indicate  how  well  the  NOLH  performs  with  respect  to  correlation: 

One  measure,  pmax,  gives  the  worst  correlation  between  design  matrix 
columns,  whereas  the  other  measure,  cond  ( XT X  ),  assesses  the  overall 
degree  of  nonorthogonality  of  the  design  matrix.  A  design  matrix  will  be 
classified  as  nearly  orthogonal  if  it  has  a  maximum  pairwise  correlation 
no  greater  than  .03  and  a  condition  number  no  greater  than  1.13.  (pp.  47) 
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Other  operations  research  studies  that  use  DOE  and  the  NOLH  include  Bodden 
(2010),  who  studies  the  survivability  of  the  transformable  craft;  Abel  (2009),  who  studies 
the  effectiveness  of  frigate  defensive  measures  in  pirated  waters;  Wegner  (2007),  who 
studies  system  reliability  and  capabilities  within  a  joint  environment;  and  Joshua  (2006), 
who  studies  the  expansion  of  the  NOLH  as  used  in  computer-based  simulations. 

3.  Design  Points 

In  modeling,  specifically  in  DOE,  input  factors  will  affect  the  results  generated  by 
the  simulation.  An  experimental  design  can  be  considered  a  mxn  matrix  where  m 
defines  the  columns  and  n  defines  the  rows  of  the  matrix.  The  in  columns  will 
correspond  to  the  input  factors  and  the  n  rows  will  correspond  to  the  factor  settings 
during  the  number  of  runs.  Each  row  represents  a  combination  of  factor-levels  and  is 
called  a  design  point  (Sanchez,  2006).  In  general,  a  proper  DOE  specifies  a  set  of  design 
points  that  efficiently  enables  the  analysis  after  the  experimental  results  are  obtained. 
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III.  U.S.  PIRACY  MODELS 


In  this  chapter,  the  first  generation  Piracy  Performance  Surface  (PPS)  and  the 
Next  generation  Piracy  Performance  Surface  (PPSN)  models  are  discussed,  as  well  as 
how  the  PPS  model  is  being  used  by  the  U.S.  and  allied  nations'  militaries,  and  the 
differences  between  the  two  piracy  models.  This  chapter  also  includes  interviews  with 
operators  and  users  of  the  PPS  model. 

A.  FIRST  GENERATION  PIRACY  PERFORMANCE  SURFACE  MODEL 

Just  days  after  the  MV  Maersk  Alabama,  a  U.S. -flagged  ship,  was  hijacked  by 
Somali  pirates  in  the  Indian  Ocean  on  April  8,  2009,  NAVO  researchers  were  tasked  by 
Rear  Admiral  David  Titley,  Oceanographer  and  Navigator  of  the  Navy,  to  assist  in  the 
fight  against  piracy  (B.  Lingsch,  personal  communication,  May  19,  20 10). 10  Admiral 
Titley  wanted  his  staff  to  define  how  meteorologists  could  play  a  vital  role  in  countering 
piracy  operations,  having  knowledge  of  the  limited  military  assets  available  to  use  and 
the  impact  of  environmental  conditions  on  the  success  of  pirate  activity.11  The  PPS 
product  was  completed  approximately  two  weeks  after  the  Maersk  incident,  and  has 
undergone  a  few  revisions  since  its  development. 

The  primary  purpose  of  PPS  is  to  be  used  as  a  piracy-forward  warning  tool  for 
military  and  commercial  vessels.  The  model  depicts  the  predicted  pirate  threat  in  SBR 
and  GOA.  Figure  5  shows  the  color  coded  output  to  the  model.  Red  represents  forecasts 
of  high  piracy  threat  against  neutral  vessels.  Areas  displayed  in  green  have  a  lower 
piracy  threat  level.  The  Sixth  Fleet  Admiral  and  his  staff  are  comfortable  with  the  PPS 
convention  in  which  red  is  indicative  of  areas  more  favorable  for  pirate  activity.12 


111  B.  Lingsch  is  head  of  the  INTEL  Department  at  the  Commander,  Naval  Meteorology  and 
Oceanography  Command  (CNMOC)  at  Stennis  Space  Center  in  Mississippi.  He  is  also  the  sponsor  of  the 
author’s  thesis  research. 

11  Dr.  James  Hansen  and  Mr.  Bill  Lingsch  in  a  presentation  to  the  Battlespace  Atmospheric  and  Cloud 
Impacts  on  Military  Operations  (BACIMO)  and  Weather  Impacts  Decision  Aids  (WIDA)  Conference  on 
April  13-15,  2010. 

12  This  information  was  gathered  during  the  author's  operations  research  experience  tour  on  May  20, 
2010.  The  author  was  given  a  presentation  by  NAVO  civilian  contract  Nathan  Hooper. 
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The  model's  index  is  a  weighted  combination  of  forecast  environmental  suitability 
for  pirate  activity  and  historical  observed  pirate  activity  with  weights  detennined  by  the 
Office  of  Naval  Intelligence  (ONI),  of  90%  and  10%  respectively  (N.  Hooper,  personal 
communication,  May  20,  20 10). 13  The  environmental  suitability  is  forecast  using  a  wave 
model  called  the  Shallow  Water  Near  Shore  model  (SWAN). 


When  a  pirate  event  in  the  form  of  a  boarding,  attempted  boarding,  or  suspected 
boarding  occurs,  an  area  surrounding  the  location  is  highlighted  to  represent  a  high-risk 
area  in  the  PPS  surface  for  the  next  forty-eight  hours,  then  slowly  dissipates  over  the  next 
seven  days.  Such  pirate  events  are  reflected  by  the  red  dots  in  Figure  5.  Historical 
attacks  provided  by  ONI  are  also  factored  into  the  pirate  activity  index  with  a  lesser 
contribution  than  the  recent  pirate  activity. 


Figure  5.  PPS  model  output  provided  to  the  CMF  assessment  team  that  was  based  on 
April  12,  2010  at  1200  a.m.  GMT.  The  areas  encircled  in  red  and  annotated  'high' 
are  areas  that  are  more  prone  to  pirate  attacks,  and  thus  should  be  areas  avoided 
by  neutral  vessels  and  monitored  by  forces  combating  piracy  in  the  SBR  and 
GOA  (From  N.  Hooper,  personal  communication,  2010). 


*3  This  information  was  gathered  during  the  author's  operations  research  experience  tour  on  May  20, 
2010.  The  author  was  given  a  presentation  by  NAVO  civilian  contractor  Nathan  Hooper. 
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B.  MILITARY  APPLICATION 


As  of  this  writing,  the  1200  Greenwich  Mean  Time  (GMT)  PPS  snapshot  is 
briefed  to  the  Commander  of  NAVCENT  at  the  Combined  Maritime  Forces 
Headquarters'  daily  meeting.14  This  snapshot  of  PPS  is  used  to  inform  the  chain  of 
command  of  the  current  day’s  relative  pirate  threat  index.  This  1200  GMT  snapshot  is 
also  provided  to  the  military  forces  deployed  to  the  GOA  and  SBR  in  support  of 
Operation  Ocean  Shield  (OOS)  and  European  Union  Naval  Force  Somalia  -  Operation 
Atalanta.  At  the  end  of  March  2011,  PPSN  will  be  briefed  at  this  meeting.  On  March  1, 
2011,  the  name  referencing  the  next-generation  piracy  model  is  the  Pirate  Attack  Risk 
Surface  (PARS). 

C.  THE  NEXT  GENERATION  PIRACY  PERFORMANCE  SURFACE 

MODEL 

The  success  of  the  PPS  model  led  the  Naval  Meteorology  and  Oceanography 
Command  (CNMOC)  leadership  to  devote  efforts  to  pursue  a  more  advanced  anti-piracy 
model,  PPSN.  Dr.  James  Hansen,  Naval  Research  Laboratory  (NRL)  Monterey,  was 
tasked  with  developing  this  next  generation  probabilistic  forecasting  tool. 

PPSN  simulates  pirate  behavior  within  the  METOC  environment  over  a  72-hour 
forecast  time  period  in  0.2-hour  time  steps  with  a  probability  surface  generated  at  12-hour 
time  steps.  The  simulated  behavior  depends  on  intelligence  parameters  such  as  the 
PAG's  camp  location,  the  PAG's  operating  area,  and  the  PAG's  mission  length.  Figure  6 
displays  the  PPSN  output,  i.e.  the  relative  forecast  of  pirate  presence  probability  as  a 
function  of  latitude,  longitude,  and  time. 

The  early  version  of  the  PPSN  model  simulated  pirates  from  randomly  generated 
starting  locations.  The  pirates  motored  to  a  pre-detennined  location  while  being  affected 
by  environmental  conditions  (waves,  winds,  and  currents),  then  drifted  until  the  pirates 
ran  out  of  simulated  food,  water,  and/or  khat,  and  then  motored  back  to  their  beginning 
location.  The  output  from  the  simulation  was  the  relative  probability  that  a  pirate  would 

14  This  information  was  gathered  on  June  8,  2010  during  the  author's  operations  research  experience 

tour. 
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be  at  a  given  latitude,  longitude  and  time.  PPSN  represented  probabilities  with  color 
coding,  with  the  highest  probabilities  in  red  and  the  lowest  probabilities  in  dark  blue. 


kmgrtude  (degrees  rasl)  longitude  (degrees  eastl 

Figure  6.  The  PPSN  model  output  displaying  the  relative  forecast  of  pirate  presence 
probability  as  a  function  of  latitude,  longitude,  and  time.  The  two  pictures 
displayed  are  the  24-hour  forecast  (left)  and  the  72-hour  forecast  (right)  of  relative 
pirate  presence  probability.  Red  corresponds  to  the  highest  probability  forecast  of 
pirate  location. 


1.  Modeling  Differences  Between  PPS  and  PPSN 


Table  1  describes  the  modeling  differences  between  the  PPS  and  PPSN  models. 


Model 

PPS 

PPSN 

1 

Overlay  of  waves,  currents  and  INTEL 
about  pirate  activity. 

Simulation  of  pirate  behavior. 

2 

Uses  METOC  forecasts  of  waves  and 

currents. 

Uses  METOC  forecasts  of  winds,  waves  and 
currents. 

3 

Assumptions  about  environmental 
suitability  based  on  a  9-foot  skiff. 

Incorporates  additional  INTEL  to  include 
pirate  METOC  knowledge,  pirate  skiff 
movement  patterns,  and  pirate  skiff  speed. 

4 

Updated  every  three  hours  unless  pirate 
activity  occurs. 

In  addition  to  updates  if  pirate  activity 
occurs,  the  model  takes  operator  inputs  that 
drive  the  model  to  include  pirate  operating 
locations,  length  of  time  a  pirate  can 
operate,  pirate  camp  locations,  and  observed 
pirate  activity. 

5 

Provides  a  snapshot  forecast  of 
environmental  suitability  for  pirate 
activity. 

Time- integrates  environment  with  pirate 
CONOPS  to  produce  relative  forecast  of 
pirate  presence  probability. 

Table  1 .  Model  comparisons  between  PPS  and  PPSN. 
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D.  INTERVIEWS  WITH  OPERATORS  AND  USERS 


The  author  traveled  to  AFRICOM  in  Stuttgart,  Germany,  and  met  with  the  CMF 
assessment  team  in  Bahrain  to  brief  them  on  the  PPS  model  and  preliminary  experiments 
using  the  MATLAB  version  of  the  PPSN  model.  The  author  met  with  users  and 
operators  to  learn  about  their  use  of  the  PPS  model,  their  needs,  and  about  pirate  tactics. 

In  Bahrain,  the  operators,  NATO  and  U.S.  Intelligence  officers,  subject  matter 
experts  (SME),  and  high  ranking  military  officers  provided  insight  and  direction  that  led 
the  author  to  make  recommendations  to  modify  the  PPSN  model  of  pirate  behavior. 
Based  on  these  recommendations,  several  changes  were  incorporated  in  the  new  version 
(Python)  of  the  PPSN  model.  Changes  that  were  based  on  the  author's  recommendations 
included:15 


•  Defining  factors  in  the  PPSN  model  that  INTEL  and 
users  justified  as  being  significantly  important  in 
PAG  CONOPS. 

•  Implementation  of  different  PAG  search  patterns 
(described  below). 

•  Redefining  the  factor  definitions  used  in  the  model  to 
assimilate  the  terminology  being  used  by  the  U.S. 
and  allied  military  operators. 

•  Implementation  of  mapping  PAG  from  their  base 
location  to  their  operating  waypoint  location. 

•  Extending  the  run  time  of  the  model  to  weeks  and 
months  rather  than  days  to  reflect  longer  PAG 
CONOPS. 


15 

thesis. 


A  trip  report  of  the  author’s  operations  research  experience  tour  is  included  in  Appendix  A  of  this 
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E.  POST  EXPERIENCE  TOUR  DESCRIPTION  OF  THE  REDEVELOPED 

PPSN 

PPSN  is  a  Monte  Carlo  probabilistic  forecasting  tool  that  takes  as  input 
environmental  conditions,  pirate  CONOPS  and  gathered  INTEL  to  produce  as  output  the 
relative  forecast  of  pirate  presence  probability  as  a  function  of  latitude,  longitude  and 
time.  PPSN  is  written  in  Python  (x,y)  version  2.6.5. 1  (Python  2.6.2).  The  Python  version 
of  PPSN  has  been  continually  in  development  since  June  2010,  and  later  experiments 
were  conducted  with  an  updated  version,  finalized  in  October  2010,  which  is  similar  (but 
not  identical)  to  the  current  operational  version. 

As  of  the  date  of  this  thesis,  PPSN  is  the  only  known  operational  Navy  product 
that  couples  METOC  and  INTEL.  It  is  used  by  the  U.S.  and  allied  forces  to  combat 
piracy  in  the  SBR  and  GOA. 

1.  Meteorology  and  Oceanography  Conditions 

The  METOC  forecast  used  in  the  PPSN  model  is  provided  by  NAVO 
oceanographic  models. 

a.  Oceanographic  Models 

The  Shallow  Water  Near  Shore  model  (SWAN)  provides  the  waves. 

The  Coupled  Ocean/Atmospheric  Mesoscale  Prediction  System  model 

(COAMPS)  provides  the  currents. 

The  Navy  Coastal  Ocean  model  (NCOM)  provides  the  surface  winds. 

b.  METOC  Parameters  Provided  by  the  User 

Input  parameters  describing  the  effect  of  METOC  on  PAGs  are  provided 
by  the  operators  that  drive  PPSN.  Typically,  this  information  will  be  provided  by  the 
U.S.  or  the  SMEs  of  the  allied  forces.  These  parameters  affect  the  PAG's  movement 
during  transit  from  a  base  (camp)  location  to  a  waypoint  location  and  when  the  PAG  is 
conducting  CONOPS  in  the  waypoint  operating  area.  The  input  parameters  listed  below 
are  defined  in  more  detail  in  Chapter  IV  of  this  thesis. 
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The  factor  by  which  the  currents  impact  the  PAG. 

The  factor  by  which  the  winds  impact  the  PAG. 

The  threshold  below  which  the  PAG  can  operate  under  certain  surface 

wind  conditions. 

The  wave  height  threshold  below  which  the  PAG  can  operate  (analogous 

to  wind  threshold). 

Whether  the  PAG  had  knowledge  of  METOC  conditions  before  leaving  its 

base  location. 

2.  Pirate  Action  Group  Mapping 

a.  Base  Mapping 

The  PPSN  model  simulates  PAGs,  which  is  a  group  of  pirate  dhows  that 
are  attached  to  a  mother  dhow  (a  larger  vessel)  that  is  typically  used  as  a  logistical  hub 
for  extending  distances  from  land  and  mission  length.  PAGs  are  given  a  specific 
beginning  location  (base)  that  can  be  a  single  point  or  a  box,  and  may  be  a  land-based 
camp  or  an  origin  point  at  a  mother  dhow  at  sea.  Each  base  has  a  longitude  and  latitude, 
and  a  longitudinal  and  latitudinal  size  of  the  box,  and  an  associated  weight.  The  weight  is 
nonnalized  in  the  code  and  defines  the  likelihood  that  the  PAG  will  use  one  base  over 
another  base  in  PPSN. 

b.  Waypoint  Mapping 

From  the  PAG's  base,  the  PAG  is  mapped  to  a  hunting  area  known  as  the 
waypoint  location.  The  waypoint  location  can  be  a  defined  by  a  single  point  or  a  box. 
The  waypoint  location  is  given  a  longitude  and  latitude,  and  a  longitudinal  and  latitudinal 
size  of  the  box  and  a  weight.  The  user  can  define  the  base  and  waypoint  to  be  the  same 
longitude  and  latitude.  This  could  potentially  be  beneficial  when  mapping  PAGs  at 
distances  1,000  nautical  miles  from  campsites.  This  technique  was  used  in  this  thesis. 

c.  PA  G  Mapping  Demonstration 

Figure  7  displays  an  example  of  the  overall  mapping  of  PAGs  from  base 
locations  to  their  respective  operating  waypoints  (annotated  by  the  arrow).  Each 
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simulated  PAG  begins  its  mission  at  a  location  randomly  sampled  from  the  base 
distribution.  For  example,  consider  the  third  yellow  box  in  Puntland  off  the  HOA.  This 
small  yellow  box  is  the  starting  base  location  that  the  operator  may  choose  to  model  as 
the  PAG's  beginning  location.  The  second  yellow  area  labeled  Distro2  may  be  the  PAG's 
waypoint.  The  PPSN  code  requires  the  lower  left-hand  comer  of  each  base  and  waypoint 
to  be  the  longitude  and  latitude  of  a  base  or  waypoint.  The  author  used  Google  Earth  to 
build  the  inputs  for  the  mapping  necessary  for  the  PPSN  model. 

When  establishing  Distro2,  the  user  will  define  this  box  by  longitude  and 
latitude  in  degrees,  (6.14°  A,  64. 86°  E1) .  The  associated  longitude  and  latitude  size  of  the 

box  is  equal  to  five  in  this  case.  A  size  of  five  for  a  box  means  approximately  a 
300x300  mile  box.  In  this  thesis,  the  author  used  a  size  of  0.01  for  the  base  boxes  and 
0.1  for  the  size  of  the  waypoints  boxes. 


Figure  7.  Representation  of  the  base  and  waypoint  locations  inputted  by  the  user  into 
the  PPSN  model  for  the  appropriate  mapping  of  each  PAG. 
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3. 


PAG  Search  Patterns 


This  factor  defines  the  movement  pattern  that  the  PAG  will  use  once  it  has 
reached  its  waypoint  longitude  and  latitude  and  begins  the  hunting  phase  (described  in 
Phase  2  below).  The  PAGs  can  be  given  any  of  the  following  search  patterns: 

•  0  (drift)  -  This  value  represents  the  PAG  drifting  as  a  function  of  the 
environmental  conditions  while  searching  for  vessels  of  opportunity. 

•  1  (random  walk)  -  This  value  represents  the  PAG  randomly  walking  within 
the  defined  longitude  and  latitude  waypoint  while  searching  for  vessels  of 
opportunity. 

•  2  (zig-zag)  -  This  value  represents  the  PAG  zig-zagging  within  the  defined 
longitude  and  latitude  waypoint.  The  zig-zag  search  pattern  resembles  the 
PAG  moving  across  a  known  shipping  transit  lane  while  searching  for  vessels 
of  opportunity. 

•  3  (transit)  -  This  represents  the  PAG  hunting  only  within  the  defined  longitude 
and  latitude  waypoint. 

4.  Three  Phases  of  PPSN 

Three  typical  PPSN  PAG  CONOPS  phases  are  as  follows: 

1 .  Phase  1  is  the  PAG's  transit  from  its  base  to  its  waypoint.  During  the  transit, 
the  PAG  will  be  impacted  by  winds,  currents,  and  wind  and  wave  threshold. 
The  PAG  will  continue  to  the  defined  waypoint  until  it  gets  within  a 
tolerance  (distance)  or  the  PAG's  mission  length  (defined  in  Chapter  IV)  has 
been  exceeded. 

2.  Phase  2  is  the  PAG’s  hunting  phase.  During  this  phase,  the  PAG  will  be 
impacted  by  winds,  currents,  and  wind  and  wave  threshold.  The  PAG  will 
hunt  using  one  of  the  described  search  patterns  until  the  PAG's  remaining 
mission  length  is  equal  to  the  time  it  took  the  PAG  to  get  to  its  waypoint 
location  (transit  time).  If  the  transit  time  is  greater  than  half  the  PAG's 
mission  length,  then  the  PAG  will  immediately  return  to  its  base. 

3.  Phase  3  is  the  transit  back  to  its  base.  During  this  phase,  the  PAG  will  be 
impacted  by  winds  and  currents,  but  will  not  be  impacted  by  wind  threshold 
and  wave  thresholds. 
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IV.  VARIABLE  DEFINITIONS 


This  chapter  describes  the  independent  and  dependent  variables  modeled  in  each 
design  of  experiments  (DOE)  in  this  thesis  to  analyze  the  PPSN  model. 

A.  INDEPENDENT  VARIABLES 

The  impacts  of  the  following  variables  were  studied  using  a  NOLH  design  to 
produce  a  robust  and  space-filling  DOE.  A  brief  description  of  each  of  the  independent 
variables  is  as  follows: 

1.  Number  of  pirate  action  groups  ( npi ):  This  discrete  factor  represents  the 
number  of  simulated  PAGs,  defined  by  the  user,  in  the  simulation.  This 
factor  must  be  an  integer  greater  than  zero. 

2.  Number  of  bases  ( nbase ) :  This  discrete  factor  represents  the  number  of 
distinct  base  locations  the  PAGs  will  originate  in  and  must  be  an  integer 
greater  than  zero.  In  this  thesis,  nbase  represents  the  total  number  of  land 
and  sea  bases. 

3.  Mission  length  (t  mission)'.  This  discrete  factor  represents  the  mean 
length  of  time  in  hours  that  PAGs  remain  active  in  the  simulation. 

4.  Number  of  PAG  replications  ( numtracks ) :  This  discrete  factor  represents 
the  number  of  times  the  model  is  replicated  for  every  PAG  in  a  given  24- 
hour  period.  For  example,  if  numtracks  equals  1,028  then  the  model  will  be 
run  1,028  times  per  PAG  for  each  24-hour  period  to  generate  the  forecast  of 
relative  pirate  presence  probability. 

5.  METOC  knowledge  ( iweather ):  This  continuous  factor  represents  the 
probability,  iweather  g[  0,  l] ,  that  PAGs  have  METOC  knowledge  prior  to 
leaving  their  base. 

6.  Skiff  speed  (skiff  _  speed ) :  This  continuous  factor  represents  the  speed 
that  a  given  PAG  is  expected  to  sustain  during  the  transit  and  searching 
patterns.  This  factor  is  measured  in  knots  and  must  be  a  positive  real 
number. 

7.  Drift  current  (drift _curr)  :  This  continuous  factor  represents  the  factor  by 
which  the  currents  impact  skiff  movement,  drift  _  curr  e  [0, 1] . 

8.  Drift  wind  (drift _wind)  :  This  continuous  factor  represents  the  factor  by 
which  the  winds  impact  skiff  movement,  drift  _  wind  e  [0, 1] . 


25 


9.  Wind  threshold  (wind  thresh) :  This  continuous  factor  represents  the 
threshold  below  which  PAGs  can  operate.  If  winds  exceed  a  PAG’s 
threshold,  the  PAG  can  no  longer  operate.  The  PAG  will  either  return  to  its 
base  or  remain  at  its  base  if  the  PAG  has  not  left.  This  factor  is  measured  in 
knots  and  is  a  positive  real  number. 

10.  Wave  threshold  (wave  _  thresh) :  This  continuous  factor  represents  the 

threshold  below  which  PAGs  can  operate.  If  waves  exceed  a  PAG's 
threshold,  the  PAG  can  no  longer  operate.  The  PAG  will  either  return  to  its 
base  or  remain  at  its  base  if  the  PAG  has  not  left.  This  factor  is  measured  in 
knots  and  is  a  positive  real  number. 

1 1 .  Pirate  search  pattern  (search  _  pattern )  :  This  categorical  factor  is  an 
integer  from  zero  to  three  that  describes  how  the  PAGs  will  conduct  their 
search  once  they  reach  their  waypoint  operating  areas.  Table  2  describes  the 
possible  PAG  search  patterns. 


Search  Pattern 

Description 

0  (drift) 

PAG  movement  is  determined  by  winds 
and  currents  only. 

1  (random  walk) 

PAGs  will  be  moving  at  the  defined 
skiff  _speed  in  a  random  walk,  plus  a 
component  due  to  winds  and  currents. 

2  (zig-zag) 

PAGs  will  be  moving  at  the  defined 
skiff  speed  in  a  zig-zag  pattern.  This 
pattern  represents  PAGs  moving  in  a 
strategic  motion  east  to  west  and  west  to 
east  across  a  southerly  to  northerly  transit 
lane.  The  zig-zag  pattern  will  also  be 
impacted  by  winds  and  currents. 

3  (transit) 

PAGs  will  be  transiting  at  the  defined 
skiff  _speed  and  will  be  impacted  by 
winds  and  currents. 

Table  2.  Descriptions  of  the  four  PAG  search  patterns.  These  patterns  take  effect 
when  the  PAGs  reach  their  waypoint  destination  and  begin  to  search  for  a  vessel 
of  opportunity. 
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B.  DEPENDENT  VARIABLES 

Summarizing  and  quantifying  the  output  from  the  PPSN  model  was  a  challenge. 
Each  simulation  run  produced  a  pirate  density  in  each  cell  at  multiple  forecast  leads 
(usually  12,  24,  36,  48,  60,  and  72  hours).  Therefore,  the  results  in  43x51  cells  at 
multiple  forecast  leads  must  be  summarized. 

In  addition,  it  was  impossible  to  use  historical  pirate  incidents  to  verily  PPSN’s 
performance  because  historical  METOC  conditions  and  available  INTEL  corresponding 
to  observed  pirate  activity  have  not  been  archived.  In  addition,  there  is  insufficient 
INTEL  gathered  from  historical  pirate  incidents  within  the  SBR  and  GOA.  Therefore, 
for  any  experiment  conducted  during  this  thesis,  there  is  no  definitive  ground  truth 
against  which  to  compare  results. 

Each  simulation  run  in  this  thesis  was  summarized  and  compared  usefully  with 
the  results  from  other  design  points  to  identify  the  variables  that  are  most  influential  and 
have  the  greatest  interaction  with  METOC  conditions.  Within  each  simulation, 
differences  across  the  forecast  lead  times  reflected  sensitivity  and  interactions  with 
METOC  conditions.  Differences  in  the  probability  fields  across  design  points  reflected 
the  influence  of  the  experimental  variables. 

A  brief  description  of  each  of  the  dependent  variables  is  given  below, 
r  =  forecast  lead  where  z  =  0,6,12,...,r  , 
z"max  =  longest  forecast  lead  provided  for  the  PPSN  METOC 
x;  =  i"‘  indexed  longitude,  where  i  =  1,...,43  and  x,  =  33 °E  and  xM  -x,  =  0.8° 

Vj  =  j"‘  indexed  latitude,  where  j  =  1,...,51  and  yI  =-15 °N  ( 1 5  5' j  and 

yj+\  -  y j  -  0-8° 

dp  =  design  point  number  from  the  DOE 
r  =  replication  number  of  each  dp 

pirate  _prob(dp,  r,i,j )  =  PPSN-forecast  of  relative  pirate  presence  probability  as 
output,  e  [0, 1] 

R  =  the  set  of  comparable  design  points 
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mean  _  pirate _  prob(z,  i,  j ,  R)  =  — ^  pirate  _  probidp ,  r,  /,  7)  e  [0, 1] 

I  ^  I  dpeR 

RMSD  and  MAXVAL  are  both  output  measurements  used  to  measure  the 
difference  between  the  forecast  of  relative  pirate  presence  probability  for  each  design 
point  and  the  mean  pirate  probability  that  is  generated  by  the  simulation. 


(1)  Root  mean  squared  difference  ( RMSD(dp,z,R ))  is  the  mean  square  root  of 

the  squared  difference  between  the  cell-by-cell  relative  pirate  probability 
pirate  _prob(dp,z,i,j) ,  and  the  mean  forecast  of  relative  pirate  presence 
probability,  mean  _  pirate  _prob(z,i,j\R) ,  for  the  set  of  comparables  R  . 

I  I  43  51 

RMSD(dp,z,R)  =  JttttZZ  (pirate  _prob(dp,  r,  i,  j )  -  mean  _  pirate _  prob( t,  i,  7,  R)f 
V  43-51  I=1  J=i 

(2)  Maximum  value  ( MAXVAL(dp,r,R ))  is  the  cellwise  maximum  absolute 

value  of  the  difference  between  the  relative  pirate  probability, 
pirate _prob(dp,r,i,j) ,  and  the  mean  forecast  of  relative  pirate  presence 
probability,  mean  _  pirate _  prob(v, i,  /,  R) . 

MAXVAL(dp,  r,  R)  =  niciX  | pirate  _  prob(dp,  r,  i,  j)  -  mean  _  pirate  _  prob(r,  i,  7,  R) | 

;=1,.. .,43,y=l,. ..,51 


These  inner  variants  measure  the  differences  across  forecasted  relative  pirate 
presence  at  different  lead  times  within  the  same  design  point.  The  following  equations 
will  define  the  inner  variants  of  RMSD  and  MAXVAL  . 

T 

‘max 

1  12 

inner  _  mean  _  prob(dp,  i,  7,  rmax)  = - ^  pirate _prob(dp,  12  •  t,  i,  7) 

max  t= 1 

12 

The  inner  variants  are  designed  to  capture  the  sensitivity  of  a  given  design  point's 
probability  field  to  METOC  conditions,  using  the  forecasts  at  different  lead  times  as  a 
proxy  for  changing  METOC  conditions. 

(3)  The  inner  variant  of  RMSD  ( iRMSD(dp,  rmax ))  is  the  square  root  of  the 

maximum  squared  difference  between  the  cell-by-cell  relative  pirate 

probability,  pirate _prob(dp,r,i, 7) ,  and  the  inner  mean  relative  pirate 

probability,  inner  _  mean  _  prob(dp,  i,  7,  rmax ) ,  of  each  design  point  over  available 

forecast  leads  (r's) . 
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iRMSD  {dp,  tmax  ) : 


1  43  51  2 

- ^  ^  max  ( pirate  _  prob  ( dp ,  r,  i,  j)  -  inner  _  mean  _  prob {dp,  i,  j,  tuax  )) 

1 43  •  5 1  i=i  j= i  *^tmax 


(4)  The  inner  variant  of  MAXVAL  ( iMA X VA  L  ( dp ,  zMAX ) )  is  the  maximum 

cellwise  absolute  difference  between  a  probability  field  and  the  inner  mean 
relative  pirate  probability,  inner  _  mean  _  prob  (dp,  i,  j,  rmax ) ,  of  each  design  point 
over  available  forecast  leads. 


iMAXVAL(dp,  rmax)  =  max  |  pirate _  prob(dp,  r,  j,  i)  -  inner  _  mean  _  prob{dp,  j,  i,  rmax) 

t<tMAX 


Smoothed  variants  of  RMSD,  MAXVAL,  iRMSD,  and  iMA  VAL  were  also  used. 
In  the  smoothed  variants,  each  cell's  value  {pirate  _  prob)  was  replaced  by  the  average 
of  the  value  plus  the  eighth  adjacent  cell’s  values  {smoothed  _  pirate  _  prob) . 

m=  j— 1 

Two  other  measures  were  also  calculated.  Mean  50th  percentile  ( MAREA  )  is  the 
average  area  containing  50%  of  the  relative  forecast  of  pirate  presence  probability 
density.  MAREA  and  the  smoothed  variant  ( MAREAs  )  were  not  used  in  the  analysis  as 
discussed  in  Chapter  VI,  Section  B. 


1  1+1 

smoothed  _  pirate  _  prob{dp,  i,  i,  j)  =  —  V 

9  k=i- 1 


7+1 

^  pirate _prob{dp,r,k,m) 
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V. 


SIMULATION  EXPERIMENTAL  DESIGNS 


This  chapter  describes  the  PPSN  experiments  used  in  this  thesis:  (A)  computer 
memory  requirements,  (B)  significant  factors,  and  (C)  variability  across  design  points. 

This  thesis  uses  the  Air  Force  Research  Laboratory  Department  of  Defense 
Supercomputing  Resource  Center  (AFRL  DSRC)  High  Performance  Computing  (HPC) 
system,  hawk,  to  perfonn  the  PPSN  simulation  runs.  The  AFRL  DSRC  HPC  is  located  at 
Wright-Patterson  Air  Force  Base  in  Ohio.  The  SGI  Altix  4700  (hawk)  consists  of  18 
computer  nodes  with  each  computer  node  having  512  processors,  all  running  a  variant  of 
the  the  Linux  operating  system  (AFRL  DSRC,  2010).  Each  of  16  computer  nodes  has  1.5 
GB  of  useable  available  memory  while  two  computer  nodes  have  3.5  GB  of  useable 
memory  (S.  Upton,  personal  communication,  February  24,  2011). 16 

This  memory  limitation  significantly  impacted  the  number  of  design  point  runs 
that  were  completed.  Steve  Upton  (personal  communication,  February  24,  2011) 
calculated  that  24,000  computer-hours  were  used  to  complete  this  thesis  on  hawk.  Upton 
(2011)  stated, 

This  may  seem  inflated;  in  order  to  use  the  large  memory  nodes  this  thesis 
needed  to  "reserve"  eight  processors  even  though  the  PPSN  model  was 
using  one  processor  in  each  of  the  jobs.  In  addition,  jobs  that  did  not 
complete  in  the  allotted  time  were  still  charged  with  that  amount  of 
computer  time.  There  were  also  a  few  trial  and  error  missteps  by  the  hawk 
user. 

A.  MEMORY  REQUIREMENT  DESIGN  OF  EXPERIMENTS 

In  attempts  to  run  the  PPSN  model,  the  computing  memory  requirements  were 
often  exceeded  resulting  in  failed  design  point  runs.  Preliminary  experiments  revealed 
that  increasing  t  _  mission  ,  numtracks ,  and  npi  significantly  increased  the  computer 
memory  requirements  of  the  PPSN  runs,  and  high  memory  requirements  could  cause 


This  information  was  provided  by  Steve  Upton,  SEED  Center  for  Data  Farming  research  associate. 
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early  termination  of  the  simulation.  Therefore,  an  experiment  was  designed  to  analyze 
each  of  the  three  memory-driving  independent  variables  separately  to  detennine  their 
influence  on  the  PPSN  output. 


1.  Mathematical  Calculation  Used  to  Determine  the  Computer  Memory 
Requirements 

Close  examination  of  the  code  showed  that  the  following  equation  approximates 
the  required  memory  for  each  design  point  (S.  Upton,  personal  communication,  February 
24,  201 1).17 


memory  _  estimate  =  ( pirate  _  location  _  size  +  METOC  _  array  _  size )  4-  2.5 

The  division  by  2.5  in  the  equation  above  is  indicative  of  the  compression  ratio  on 
Microsoft  Windows  software. 
ntaus  =  the  number  of  forecasts 
max(7  _  mission) 


hrs  = 


nw  = 


+  ntaus 


2  *  max (7  _  mission ) 
24 


METOC  array  size  =  (3 * 201*171*2 */tra) +  (2*321* 273 *2* hrs) 

The  values  171,  201,  273,  and  321  are  fixed  array  sizes  based  on  the  array  size  of  the 
provided  METOC  data. 


pirate  _  location  _  size  =  npi  x 


npi  ( nw(numtracks  + l))  * 


|724*21*i 

'  (ntaus- 1)*6^ 

[  dt  J 

t  _  mission 

-(24*(«w  +  l)) 


Using  this  memory  calculation  allowed  the  author  to  create  design  of  experiments 
that  could  be  run  in  a  reasonable  operational  time  and  produce  relevant  output  results 
with  less  than  the  approximately  24  GB  (16*1 .5GB )  of  memory  available. 


17  This  formula  was  provided  by  Mr.  Steve  Upton,  research  associate  for  the  SEED  Center  for  Data 
Farming. 
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2.  Description  of  Three  Independent  Variables  That  Increase  Computer 

Memory  Requirements 

The  memory  capacity  of  about  24  GB  precluded  using  a  space-filling  design  that 
varied  t  mission ,  numtracks ,  and  npi  throughout  realistic  ranges. 

Appendix  B  displays  the  values  given  to  each  design  point.  This  appendix  also 
shows  the  three  cases  modeled  in  this  DOE.  Design  point  one  is  used  as  the  base  case 
design  point  for  this  DOE  (refer  to  Appendix  B).  Design  point  one  will  remain  constant 
through  the  simulations  and  when  analyzing  the  three  memory  requirements  variables. 
Design  point  one  has  the  following  values  associated  with  each  variable:  t  _  mission  =  36 , 
numtracks  =128,  and  npi  =  4.  The  computer  memory  requirement  for  this  base  case 
was  less  than  1  GB. 

a.  The  Number  of  PA  G  Replications 

The  first  sub-design,  design  points  1  and  2-16,  varies  numtracks  while 
holding  t  mission  and  npi  constant.  The  input  values  given  to  numtracks  wary 
between  128  and  2,048  PAG  replications  per  24-hour  period  in  increments  of  128  PAG 
replicates.  This  variable  is  used  in  the  PPSN  model  to  produce  the  forecast  of  relative 
pirate  presence  probability.  This  variable  is  not  reflective  of  gathered  INTEL  from  the 
military  assets  countering  piracy  in  the  SBR  and  GOA.  It  is  solely  used  for  modeling 
purposes. 


b.  Mission  Length 

The  second  sub-design,  design  points  1  and  17-35,  varies  t  mission 
while  numtracks  and  npi  are  held  constant.  The  input  values  given  to  t  _  mission  vary 
between  24  and  336  hours  in  increments  of  6,  12,  and  24  hours,  respectively  (refer  to 
Appendix  B).  Despite  336  hours  being  the  longest  t  mission  in  this  DOE,  the  CMF 
assessment  team  INTEL  officer,  LCDR  K.  Lutz,  (personal  communication,  June  8,  2010) 
stated  that  the  increase  in  pirate  capabilities  has  enabled  the  PAGs  to  sustain  pirating 
missions  for  longer  than  336  hours;  thus,  336  hours  is  a  moderately  low  mission  length 
for  advanced  PAG  CONOPS. 
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c.  The  Number  of  PA  Gs 

The  third  sub-design,  design  points  1  and  36-46,  varies  npi  while 
t  mission  and  numtracks  are  held  constant.  The  input  values  given  to  npi  vary 
between  4  and  48  in  increments  of  four  PAGs.  The  current  operational  testing  being 
conducted  on  PPSN  has  injected  real-time  INTEL  reflecting  the  value  associated  with 
this  variable. 

B.  SIGNIFICANT  FACTORS  DESIGN  OF  EXPERIMENTS 

The  following  DOE  with  33  design  points  investigates  which  PPSN  input 
parameters  are  the  most  influential  in  determining  the  distribution  of  relative  risk  of 
pirate  presence  in  PPSN  and  which  PPSN  input  parameters  interact  most  strongly  with 
METOC  conditions  and  INTEL  parameters. 

Table  3  shows  the  variables  modeled  and  the  ranges  of  values  given  to  each  of  the 
variables  discussed  in  Chapter  IV.  With  the  exception  of  t  _  mission  ,  these  values  are 
realistic  according  to  information  gained  during  the  author's  operations  research 
experience  tour. 


Independent  Variables 

Minimum  Value 

Maximum  Value 

npi 

10 

30 

t  _  mission 

72 

240 

numtracks 

85 

888 

skiff  _  speed 

5 

35 

drift  curr 

0.5 

1.0 

drift  _  wind 

0.4 

0.8 

wind  _  thresh 

20 

30 

wave  _thresh 

3 

9 

iweather  ( probability ) 

0 

1 

search   pattern(categorical ) 

— 

0,1, 2, 3 

Table  3.  NOLH  maximum  and  minimum  values  for  the  ten  independent  variables 
used  in  the  significant  factors  design  of  experiments. 
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In  addition,  skiff  _speed  standard  deviation  was  set  to  20%  of  each  design 
point's  skiff  _speed  .  The  wave _thresh  standard  deviation  was  set  to  a  NOLH  design 
variable  with  values  ranging  from  0.1  to  0.9  multiplied  by  the  value  of  wave  _thresh  . 

Since  npi  ranges  from  10  to  30,  the  PAGs  are  assigned  to  the  six  bases  randomly. 
One-third  are  assigned  to  land  bases,  which  are  sampled  without  replacement  until  all 
land-base  positions  have  been  used  at  least  once,  and  then  with  replacement  until  all 
PAGS  have  been  assigned  a  base.  Each  land  base  is  paired  with  a  hunting  waypoint.  The 
remaining  sea-based  PAGs  are  also  assigned  to  the  three  sea  bases  in  the  same  way.  For 
sea-based  pirates,  their  base  is  identical  to  their  hunting  waypoint. 

A  NOLH  was  used  to  achieve  near-orthogonality  and  a  good  space-fdling  design 
for  the  10  independent  variables.  Appendix  C  provides  the  full  NOLH  design  of  the  33 
design  points.  Due  to  the  memory  and  runtime  requirements,  a  rotational  design,  or  other 
larger  designs,  was  not  feasible  for  this  experiment.  The  runs  which  completed  indicated 
that  runtime  was  highly  correlated  with  memory  requirements  in  GB  (refer  to  Figures  8 
and  9). 

Figure  8  represents  the  original  DOE  where  the  memory  requirements  for  the  16 
design  points  that  did  not  complete  (level  zero)  ranged  from  0.7  to  15.8  GB.  These 
design  points  are  3,  5,  6,  11,  12,  14,  15,  16,  17,  18,  21,  24,  25,  26,  27  and  30.  Design 
point  18  has  the  highest  total  memory  requirement,  15.79  GB.  This  design  point  also  has 
a  drifting  search  _  pattern .  For  the  design  points  that  completed  (level  one),  design 
points  1  and  2  are  the  outliers  with  memory  requirements  of  14.7  GB  and  14.3  GB, 
respectively.  This  difference  between  these  two  design  points  and  design  point  18  is  their 
search  _  pattern  .  Design  points  1  and  2  had  a  transit  only  search  _  pattern  . 
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Figure  8.  Original  DOE  with  33  design  points.  A  level  of  one  indicates  design  points 
That  ran  to  completion  and  zero  indicates  design  points  that  failed  to  complete. 


In  order  for  the  remaining  16  design  points  to  run  to  completion,  the  DOE  was 
modified  (Figure  9).  The  length  of  t  mission  was  decreased  by  20%  of  its  original 
value,  numtracks  was  decreased  by  60%  of  its  original  value,  and  the  allotted  runtime 
was  increased  from  30  hours  to  60  hours.  Decreasing  t  mission  and  numtracks 
reduced  the  remaining  design  points'  memory  requirements  to  less  than  9.5  GB. 


Figure  9.  Modified  DOE  with  33  design  points.  The  design  points  that  did  not  complete 
(level  0)  had  a  reduction  in  t  _  mission  and  numtracks  . 
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The  scatterplot  matrix  (Figure  10)  shows  that  the  10  independent  variables  used  in 
the  significant  factors  DOE  have  nearly  linear  independence,  even  after  the  modification 
of  the  DOE.  The  correlation  between  the  variables  is  less  than  0.3. 


Scatterplot  Matrix 


wind  wave_thres  search_ 

t_mission  skiff_speeri  rlrift_curr  drift_wind  _thresh  h  weather  pattern  npi 


Figure  10.  Scatterplot  matrix  of  the  significant  factors  DOE. 


C.  VARIABILITY  ACROSS  DESIGN  POINTS 

Long  runtimes  prevented  conducting  a  larger  experiment  with  many  replications; 


hence,  a  small  experiment  was  conducted  to  evaluate  the  variability  inherent  in  the  PPSN 
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model.  The  question  that  this  experiment  addresses  is  how  much  variation  is  there  for  a 
single  design  point  due  to  the  stochastic  variation  in  the  model.  A  single  design  point 
from  the  significant  factors  DOE  is  chosen — design  point  13.  This  design  point  is  chosen 
because  it  is  not  an  outlier  and  its  memory  requirement  is  less  than  3.5  GB,  0.9  GB  to  be 
precise.  Design  point  13  is  replicated  23  times  with  the  variable  values  as  defined  in 
Table  4. 


Independent  variables 

Value 

npi 

16 

t  _  mission 

172 

man  tracks 

100 

skiff  _  speed 

10.63 

drift  _  curr 

0.78 

drift  _  wind 

0.62 

wind  _  thresh 

21.25 

wave  _thresh 

8.81 

iweather 

0.78 

search  _  pattern 

1 

Table  4.  Design  point  13  values  for  the  variability  across  design  points  DOE. 
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VI.  DATA  ANALYSIS 


The  previous  five  chapters  discussed  the  problems  of  piracy,  the  two  piracy 
performance  surface  models,  the  independent  and  dependent  variables  used  in  this  thesis, 
and  the  simulation  DOEs. 

In  this  chapter,  the  results  from  each  of  the  experiments  are  analyzed.  The 
conclusions  drawn  from  this  analysis  are  used  for  refining  INTEL  inputs  and  PPSN 
developmental  efforts. 

A.  METHODOLOGY  AND  TOOLS 

For  each  forecast  lead  time  and  design  point,  a  picture  in  the  form  of  a  jpeg  file  is 
produced,  which  shows  the  forecast  of  relative  pirate  presence  probability  (see  Figure  6 
for  examples  in  Chapter  III,  Section  C). 

1.  Analytical  Tool 

JMP  is  used  as  the  statistical  analysis  software  in  this  thesis.18  JMP's  robust  tool 
sets  allow  for  the  use  of  one  software  program  to  apply  the  following  analytical  methods 
used  here:  (1)  partition  trees,  (2)  stepwise  regression  methods,  (3)  sensitivity  analysis, 
and  (4)  multivariate  correlation. 

2.  Linear  Regression  Analysis 

This  thesis  will  use  the  following  multivariate  regression  equation  for  determining 
the  "best"  fit  model: 

k  k  k  j 

y  =  A  +  X  Pixt + X  Pt/i  +  X  X  Pijxixj + £ 

i= 1  i=\  ]=\  i=l  ’ 

where  s  =  error  . 

Montgomery,  Peck,  and  Vining  (2006)  discuss  three  stepwise-type  procedures  for 
analyzing  data.  These  broad  categories  include  forward  selection,  backward  elimination, 
and  stepwise  regression.  This  thesis  uses  forward  selection  which  assumes  no 


18  JMPR ,  Version  9.  SAS  Institute  Incorporated,  Cary,  N.C.  Retrieved  from  http://www.jmp.com. 
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independent  variables  are  initially  in  the  model.  The  first  variable  to  be  included  in  the 
model  is  the  independent  variable  with  the  greatest  correlation  to  the  dependent  variable. 
Additional  independent  variables  are  added  in  the  model  if  their  partial  correlation  is 
significant  after  adjusting  for  the  effects  of  the  included  independent  variables. 

The  significance  levels  for  variables  to  enter  and  exit  the  regression  model  are 
PENTER  =  0.05  and  PEXIT  =0.10.  Devore  (2009)  states  the 

P  -value  ( observed  significance  level)  is  the  smallest  level  of  significance 

at  which  H0  would  be  rejected  when  a  specified  test  procedure  is  used  on 

a  given  data  set.  Once  the  P  -value  has  been  determined,  the  conclusion 

at  any  particular  level  a  results  from  comparing  the  P  -value  to  a  : 

1.  P -value  <  a  =>  reject  H0  at  level  a  . 

2.  P -value  >  a  =>  do  not  reject  H0  at  level  a  (pp.  313). 

The  null  hypothesis,  H0 ,  is  the  hypothesis  assumed  to  be  true,  and  the  test  procedures  are 
an  attempt  to  disprove  the  null  hypothesis.  Devore  (2009)  states  "the  two  possible 
conclusions  from  a  hypothesis-testing  analysis  are  then  reject  H0  or  fail  to  reject  H0" 
(pp.  285). 

The  coefficient  of  determination,  R2 ,  is  used  as  one  of  the  key  measures  of  model 
effectiveness  (Montgomery  et  ah,  2006).  R 2  is  the  proportion  of  total  variance  explained 
by  the  model.  A  R2  value  close  to  1  means  most  of  the  variation  in  the  dependent 
variable  is  explained  by  the  regression  equation. 

3.  Classification  and  Regression  Trees 

The  use  of  classification  and  regression  trees  can  be  a  powerful  tool  for 
understanding  results  graphically.  The  tree  methods  explain  the  variation  of  the  output 
variable  (dependent  variable)  by  splitting  or  partitioning  on  the  data  recursively,  creating 
homogeneous  branches  until  a  stopping  criterion  is  met.  The  stopping  criteria  for  this 
thesis  use  the  sum  of  squares  for  continuous  variables  and  the  log-likelihood  chi-squared 
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for  discrete  and  categorical  variables.19  A  regression  tree  is  used  for  this  thesis  to  show 
clear-cut  partitions  for  the  influences  of  the  independent  variables  modeled  in  PPSN. 


B.  MULTIVARIATE  CORRELATION 

Chapter  IV  described  each  of  the  ten  output  metrics.  The  multivariate  correlation 
scatterplot,  depicted  in  Table  5,  shows  the  pairwise  correlations  among  the  dependent 
variables  resulting  from  the  significant  factors  experiment.  The  high  correlation  among 
RMSD ,  MAXVAL  and  their  smoothed  versions  indicates  that  only  one  of  these  four 
dependent  variables  needs  to  be  used  in  the  analysis  of  this  thesis.  Thus,  RMSD  is  the 
first  dependent  variable  used  in  the  analysis.  Furthermore,  high  correlation  exists  among 
iRMSD  and  iMAXVAL  and  their  respective  smoothed  versions.  Therefore,  iRMSD  is  the 
second  dependent  variable  used  in  the  analysis. 

Table  5  also  shows  that  MARE  A  and  MARE  As  are  negatively  correlated  to  each 
of  the  other  eight  dependent  variables  and  positively  correlated  to  each  other.  However, 
these  dependent  variables  did  not  significantly  depend  on  any  of  the  DOE  input  variables. 
Therefore,  these  output  variables  are  not  used  in  this  thesis. 


19  Details  on  how  JMP  calculates  the  p  —  value  using  partition  tree  analysis  can  be  found  at 
http://www.jmp.com/blind/whitepapers/wp_montecarlo_cal.pdf. 
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Table  5.  Multivariate  correlation  scatterplot  matrix  of  the  output  statistics  for  the 
24-hour  forecast  from  the  significant  factors  experiment  with  33  design  points. 
This  experiment  was  used  to  test  the  correlation  between  the  output  metrics. 
RMSD  and  iRMSD  are  the  two  dependent  variables  used  in  this  thesis. 


C.  RESULTS 

1.  Computer  Memory  Requirement  Experiment 

The  computer  memory  requirement  experiment  analyzes  the  three  memory 
variables,  t  _  mission  ,  numtracks ,  and  npi ,  to  pinpoint  how  these  variables  affect  the 
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PPSN  output.  The  underlying  question  that  this  experiment  answers  is  whether  the  value 
given  to  these  variables  alters  the  forecast  of  relative  pirate  presence  probability.  The 
information  gathered  from  this  experiment  is  important  for  the  operational  testing  being 
conducted  on  PPSN.20 


a.  Mission  Length 

The  modelers  are  using  a  mission  length  of  seven  days  (168  hours)  in 
operational  testing  being  conducted  on  PPSN.  D.  Laleijini21  stated,  "We  want  to  keep  the 
PAGs  in  the  model  for  as  long  as  we  reasonably  can  based  on  processing  time  and 
memory  requirements"  (personal  communication,  February  7,  2011).  The  following 
t  _  mission  analysis  will  show  why  special  precautions  should  be  taken  when  decreasing 
t  mission  to  meet  operational  objectives. 

Figure  1 1  shows  the  results  of  varying  t  _  mission  values  together  with  the 
best  fit  quadratic  equation  and  reveals  that  t  mission  does  matter.  What  Figure  1 1  does 
not  show  is  what  specific  values  of  t  _  mission  should  be  used.  It  may  appear  that  the 
knee  of  the  curve  lies  between  a  t  mission  value  of  150  and  225  hours.  This  is  an 
incorrect  assumption.  These  results  do  not  show  that  values  of  t  _  mission  produce  the 
best  results.  These  results  show  that  small  values  of  t  mission  will  produce  very 
different  results  than  increased  values.  A  t  mission  value  that  reflects  true  PAG 
CONOPS  should  be  used  rather  than  a  t  _  mission  that  gives  PPSN  a  shorter  run  time. 

The  military  operator  conducting  missions  in  the  SBR  requires  an  accurate 
portrayal  of  the  relative  pirate  presence  probability.  Unlike  PPS,  PPSN  provides  better 
information  that  focuses  the  military's  surface  assets  to  a  specific  area  within  the  SBR. 
Since  the  SBR  covers  millions  of  square  miles  using  realistic  CONOPS  for  simulated 
PAGs  is  important.  Decreasing  a  PAG's  mission  length  will  produce  very  different  PPSN 
output  results. 

20  Effective  March  1,  201 1,  the  name  of  the  next-generation  Piracy  Performance  Surface  model, 

PPSN,  was  changed  to  the  Pirate  Attack  Risk  Surface  model,  PARS. 

21  D.  Lalejini  is  a  civilian  researcher  for  NRL-Stennis  Space  Center  in  Mississippi. 
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Figure  11.  Effect  of  varying  t  _mission  on  RMSD  for  the  72-hour  forecast  probability 
field  together  with  the  best  fit  quadratic  model.  The  model's  i?2  is  0.57. 

b.  Number  of  Pirate  Action  Group  Replications 

The  number  of  PAG  replications,  numtracks ,  is  another  variable  that 
significantly  impacts  computer  memory  requirements.  The  variable  numtracks 
determines  how  many  times  the  PPSN  model  is  replicated  with  the  same  input  variables 
to  produce  the  pirate  probability  fields. 

Figure  12  shows  the  result  of  varying  numtracks .  A  smaller  value  of 
numtracks  produces  a  larger  difference  between  each  design  point  and  the  mean 
probability.  Figure  12  shows  that  RMSD  is  influenced  by  numtracks  over  the  entire 
experimental  range  from  128  to  2048. 
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Figure  12.  Effect  of  varying  numtracks  on  RMSD  for  the  72-hour  forecast  probability 
field  together  with  the  best  fit  quadratic  model.  The  model's  R 2  is  0.757. 

Initially,  the  operational  testing  being  conducted  on  PPSN  was  using 
numtracks  equal  to  512.  After  a  discussion  with  the  developer  and  modelers,  numtracks 
was  increased  to  1024  replications. 

In  closing,  this  variable  should  be  hard-coded  into  the  PPSN  model  rather 
than  letting  the  operator  set  the  value  since  it  is  influential  but  it  is  not  impacted  by 
changing  METOC  or  INTEL. 

c.  Number  of  Pirate  Action  Groups 

The  current  operational  testing  being  conducted  on  PPSN  uses  a  value  of 
npi  that  is  based  on  recent  anti-shipping  activity  messages  from  the  National  Geospatial- 
Intelligence  Agency  (NGA)  indicating  when  a  PAG  location  has  been  reported  (D. 
Lalejini,  personal  communication,  February  7,  201 1).  The  observed  PAGs  remain  in  the 
PPSN  model  for  a  maximum  of  six  days.  A  typical  value  is  npi  =  6 ,  which  is  lower  than 
most  values  used  in  the  significant  factors  experiment  of  this  thesis.  Those  conducting 
the  operational  testing  include  only  reported  PAGs,  but  they  are  currently  unable  to 
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acquire  the  most  accurate  and  real-time  intelligence  on  current  PAG  activity  in  the  SBR 
and  GOA.  The  true  number  of  P AGs  operating  at  one  time  is  believed  to  be  greater. 

Figure  13  shows  npi  ranging  between  8  and  48  PAGs.  The  author's 
operations  research  experience  tour  interviews  reveal  that  limited  knowledge  is  available 
on  the  exact  number  of  PAGs  operating  within  (on  the  high  seas)  the  SBR  and  GOA  and 
along  the  coastline  at  camp  sites.  For  this  reason,  a  wide  range  of  scenarios  for  npi 
should  be  considered.  Figure  13  shows  that  PPSN  output  is  sensitive  to  the  value  of  npi 
throughout  the  experimental  range,  but  the  effect  of  npi  diminishes  substantially  around 
npi  =  30 .  At  this  point,  it  should  be  noted  that  for  all  design  points  in  this  thesis 
experiment,  the  same  four  bases  and  four  hunting  waypoints  are  used  for  all  values  of 
npi ,  so  differences  across  design  points  are  due  to  the  number  of  PAGs,  not  the  number 
or  locations  of  bases  and  hunting  waypoints. 


Figure  13.  Effect  of  varying  npi  on  RMSD  for  the  72-hour  forecast  probability  field 
together  with  the  best  fit  quadratic  model.  The  model's  R2  is  0.944. 

In  conclusion,  this  variable  should  be  controlled  by  the  operator.  Specific 
intelligence  should  be  gathered  as  to  provide  realistic  values  for  this  variable. 
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2.  Significant  Factors  Experiment 

The  significant  factors  experiment  is  analyzed  using  partition  trees  and  linear 
regression.  The  ten  independent  variables  this  DOE  uses  were  selected  based  on 
information  collected  during  the  author's  operations  research  experience  tour  to  the 
Combined  Maritime  Force's  headquarters  in  Bahrain. 

The  DOE  answers  the  following  two  questions: 

1 .  Which  PPSN  input  parameters  are  the  most  influential  in  determining  the 

distribution  of  relative  risk  of  pirate  presence  in  the  PPSN? 

2.  Which  PPSN  input  parameters  interact  most  strongly  with  METOC 

conditions  and  INTEL  parameters? 

Infonnation  gathered  from  this  DOE  will  help  the  developer,  modelers,  and 
operators  in  focusing  their  efforts  towards  a  more  direct,  systematic  approach  when  using 
PPSN  and  gathering  INTEL  on  PAG  CONOPS. 

a.  Significant  Factors  Using  Partition  Trees 

In  a  partition  tree,  the  splits  indicate  the  best  predictors  of  the  output 
metrics.  The  partition  tree  for  RMSD  shows  which  METOC  and  INTEL  variables  have 
the  greatest  effect  on  the  PPSN  output,  while  the  iRMSD  partition  tree  shows  how 
variables  interact  with  METOC  changes  across  the  forecast  leads. 

Figure  14  shows  that  the  majority  of  the  variation  in  RMSD  is  explained 
by  search  _  pattern .  Recall,  the  following  search  _  pattern  definitions: 

•  0  (drift) 

•  1  (random  walk) 

•  2  (zig-zag) 

•  3  (transit) 
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Figure  14.  Partition  tree  displaying  a  R  value  of  0.58.  The  dependent  variable  is 
RMSD  where  z  =  72  hours 


One  of  the  important  results  of  the  author's  operations  experience  tour  was 
the  addition  of  search  _  pattern  in  the  PPSN  code.  The  developer  initially  only  had  the 
drifting  search  pattern;  three  additional  search  patterns  were  implemented.  Figure  14 
shows  that  search  _  pattern  is  highly  significant  in  influencing  the  PPSN  output.  The 
INTEL  community  should  concentrate  their  efforts  on  determining  PAG  search  behavior. 
This  knowledge  will  prove  to  be  beneficial  for  the  PPSN  output. 

The  variability  chart  for  RMSD  at  72  hours  (Figure  15)  shows 
search  _  pattern  equal  to  3  (transit  only)  and  reflects  the  greatest  difference  in  variability 
in  the  PPSN  model.  This  means  that  a  PAG  modeled  with  a  transit  only  searching  pattern 
has  the  greatest  difference  in  pirate  probability  fields  compared  to  the  mean  pirate 
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probabilities.  As  stated  previously,  specific  INTEL  should  be  gathered  to  determine  if 
PAGs  use  a  transit  only  searching  pattern  when  looking  for  a  target  vessel. 


Figure  15.  Variability  chart  for  RMSD  at  72  hours 

The  second  partition  occurs  on  drift _wind  at  0.2.  Recall  that 
drift  _  wind  is  a  continuous  variable  between  0  and  1  that  represents  the  factor  by  which 
winds  impact  the  PAGs'  movement.  Another  way  to  look  at  this  is  that  it  is  important  to 
know  how  much  of  PAGs'  movement  is  affected  by  surface  winds  in  order  to  predict  the 
relative  distribution  of  pirate  presence  probability.  The  third  partition  in  Figure  14  also 
occurs  on  drift  wind .  This  time  the  split  occurs  on  a  factor  of  0.6128.  This  result 
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means  that  drift _wind  above  0.2  and  below  0.6128  leads  to  output  similar  to  the  mean 
pirate  probability  fields,  but  drift _wind  below  0.2  or  above  0.6128  produces  results 
different  from  the  mean  pirate  probability  fields. 

The  inner  metric,  iRMSD ,  captures  the  sensitivity  of  the  probability  field 
within  each  design  point  to  a  change  in  the  forecast  leads.  Figure  16  shows  a  partition 
tree  using  iRMSD  as  the  dependent  variable.  The  first  and  second  splits  occur  on 
search  _  pattern  indicating  that  a  search  pattern  of  3  (transit)  or  0  (drift)  makes  the 
design  point  more  sensitive  to  METOC.  The  third  split  occurs  on  wind  thresh  .  Recall 
that  wind  thresh  is  a  continuous  factor  that  represents  the  wind  speed  threshold  below 
which  PAGs  can  operate.  If  winds  exceed  a  PAG's  threshold,  the  PAG  will  either  return 
to  its  base  or  remain  at  its  base  if  it  has  not  left. 

INTEL  should  be  gathered  for  this  specific  variable  depending  on  the 
change  in  the  METOC  across  each  season.  Since  this  variable  describes  much  of  the 
variability  in  RMSD ,  associating  a  realistic  value  is  important.  A  recommendation  is  to 
have  the  modelers  at  ONI  input  the  wind  threshold  and  standard  deviation. 

The  final  split  occurs  on  npi ,  the  number  of  PAGs  simulated  in  PPSN.  As 
described  previously  in  Section  C.l  of  this  chapter,  better  intelligence  should  be  gathered 
on  the  value  given  to  this  variable. 
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Figure  16.  Partition  tree  displaying  a  R 2  value  of  0.47.  The  dependent  variable  is 
iRMSD . 


b.  Significant  Factors  Using  Linear  Regression 

Linear  regression  is  the  second  way  this  experiment's  results  are  analyzed. 
This  thesis  uses  linear  and  quadratic  terms,  and  first-order  interaction  terms  as  shown  in 
Chapter  VI,  Section  A.2  in  the  linear  regression  equation. 

(1)  RMSD  as  a  Measure  of  Output.  The  linear  regression 
model's  summary  of  fit  (Figure  17)  shows  that  the  model  generated  by  the  stepwise 
regression  has  a  R 2  value  of  98.4  percent.  This  R2  is  nearly  1,  meaning  that  most  of  the 
variation  in  the  model  is  explained  by  the  regression  equation. 
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Figure  17.  Summary  of  fit  and  analysis  of  variance  for  the  linear  regression  analysis  for 
RMSD . 


Figure  18  shows  that  seven  of  the  ten  independent  variables 
modeled  in  this  DOE  are  significant  in  determining  RMSD  for  a  forecast  lead  of  72 
hours.  Notice  that  unlike  the  partition  tree,  drift  wind  is  not  included  in  the  linear 
regression  model.  This  result  is  a  product  of  the  partition  tree  recursively  splitting  on 
homogeneous  groups  while  the  linear  model  uses  a  regression  equation. 

Also,  take  note  of  the  interaction  between  search  _  pattern  and 
iweather  .  Recall,  iweather  is  the  probability  that  PAGs  have  METOC  knowledge  prior 
to  leaving  on  a  mission.  This  result  is  interesting  because  PAGs  are  given  a  specific 
search  _  pattern  and  iweather  probability  before  the  simulation  spin-up.  Both  of  these 
variables  are  highly  influential.  Therefore,  operators  should  focus  their  efforts  on  getting 
valid  INTEL  about  these  two  variables.  A  PAG  having  knowledge  of  METOC  is  going 
to  mitigate  the  effect  of  the  PAG's  search  pattern  on  the  forecast  of  relative  pirate 
presence  probability. 

In  Figure  18,  the  two  tenns  with  quadratic  effects  are  t  mission 
and  iweather  .  These  two  squared  tenns  show  that  these  two  variables  have  a  nonlinear 
effect.  The  variable  iweather  should  be  estimated  based  on  intelligence  gathered  of  the 
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sophistication  at  each  camp  site.  This  variable  does  interact  with  search  _  pattern ,  the 
variable  that  was  shown  in  the  partition  tree  to  describe  most  of  the  variability  in  the 
model. 


Another  interesting  aspect  of  Figure  18  is  that  the  three  computer 
memory  requirement  variables  discussed  earlier  in  this  chapter  are  significant.  As 
discussed  previously,  care  must  be  taken  when  selecting  values  for  these  three  variables 


as  they  are  highly  influential. 
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Figure  18.  Model  parameter  estimates  overview  for  RMSD  at  72  hours. 


(2)  iRMSD  as  a  Measure  of  Output.  The  output  metric, 
iRMSD ,  captures  the  sensitivity  of  a  given  design  point  to  METOC  conditions,  using  the 
forecasts  at  different  lead  times  as  a  proxy  for  changing  METOC  conditions.  Figure  19 
provides  the  sorted  parameter  estimates;  this  reveals  the  independent  variables  that  are 
significant  and  influential. 

One  interaction  term  that  has  not  been  seen  in  any  of  the  previous 
analyses  is  skiff  _  speed  and  wind  _  thresh  .  Interactions  among  variables  can  often  be 
difficult  to  interpret.  When  analyzing  interactions,  one  area  of  focus  is  each  variable's 
parameter  or  (3 .  The  [3  value  allows  the  analyst  to  determine  the  effect  of  each  variable 
and  their  interaction  on  the  model. 

For  example,  this  linear  model  includes  the  following  terms: 

R  v  3  X  -4 -  3  X  X 

/  wind _ thresh  wind _ thresh  /skiff _ speed  skiff _ speed  /  wind _ thresh : skiff _ speed  wind _ thresh  skiff _ speed 
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This  equation  implies  that  the  effect  of  a  change  in  skiff  _  speed  is  dependent  on  the 
value  of  wind  thresh  .  Likewise,  the  effect  of  a  change  in  wind  _thresh  is  dependent 
on  the  value  of  skiff  _  speed  . 


Figure  19  shows  that  t  mission  appears  as  the  only  nonlinear 
term  (other  than  interaction  terms).  As  stated  previously,  the  value  of  t  mission  is 
important  when  modeling  PAG  CONOPS  in  the  PPSN  model. 
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Figure  19.  Significant  factors  sorted  model  parameter  estimates.  R 2  =  0.99  . 


3.  Variability  Experiment 

The  variability  experiment  is  designed  to  see  how  the  variability  for  a  single 
design  (design  point  13)  point  replicated  23  times  compares  with  the  variability  across 
design  points  in  the  significant  factors  experiment.  Table  6  shows  the  ratio  of  the 
standard  deviation  of  RMSD  for  the  variability  experiment  (<JVAR)  to  the  standard 

deviation  of  RMSD  for  the  significant  factors  experiment  ( crSIG  FACTORS)  for  three 

forecast  leads.  For  each  forecast  lead,  the  ratio  is  less  than  16%.  This  metric  provides 
evidence  that  the  variation  in  the  significant  factors  experiment  is  primarily  due  to  the 
variable  settings  in  the  DOE,  rather  than  the  variation  created  within  the  PPSN 
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simulation.  So,  for  example,  if  one  operator  were  to  run  the  simulation  with  the  same 
input  variables  and  values  more  than  once,  the  operator  should  achieve  similar  forecasts 
of  relative  pirate  presence  probability. 


^  VAR 

®  SIG  _  FACTORS 

24  hour  RMSD 

0.157 

48  hour  RMSD 

0.109 

72  hour  RMSD 

0.070 

Table  6.  The  ratio  between  the  standard  deviation  of  design  point  13  from  the 
significant  factors  experiment  and  the  33  design  point  significant  factors 
experiment.  The  output  measurement  is  RMSD  for  forecast  leads  of  24,  48,  and 
72  hours. 

The  results  of  three  carefully  designed  simulation  experiments  identify  the  most 
important  drivers  of  the  PPSN  output  and  their  significance.  The  results  of  this  thesis 
have  also  provided  insights  that  have  been  and  can  be  used  to  improve  parameter 
selection  for  operational  implementation  of  the  PPSN  model  to  allow  quick  turnaround  of 
updates  to  the  product  in  response  to  new  intelligence,  within  memory  constraints,  and 
without  sacrificing  product  quality. 
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VII.  CONCLUSION  AND  RECOMMENDATIONS 


A.  RESEARCH  SUMMARY 

This  thesis  research  includes  visits  and  interviews  with  users  of  PPS  and  other 
counter-piracy  forces  to  understand  the  operators'  needs  and  to  gain  intelligence  on  the 
most  current  pirate  CONOPS  within  the  SBR  and  GOA.  This  gathered  intelligence  led  to 
recommended  changes  in  the  PPSN22  code.  In  addition,  using  robust  experimental 
designs,  this  research  identifies  the  most  significant  INTEL  factors  driving  the  PPSN 
output.  This  gathered  information  is  being  used  by  the  developer  and  modelers  for  the 
refining  of  these  input  variables  and  choosing  the  best  functionality  for  enhancements  to 
the  PPSN  model.  This  research  also  unveils  input  variables  that  are  influential  in  the 
computing  memory  requirement  and  runtime.  This  information  is  being  used  to  focus 
efforts  on  setting  these  variables  at  realistic  levels,  without  sacrificing  the  model's 
efficiency  and  effectiveness,  to  allow  for  quick  turnaround  of  updates  to  the  PPSN  model 
in  response  to  gathered  INTEL. 

B.  RESEARCH  QUESTIONS 

This  thesis  addresses  the  following  questions: 

1.  What  improvements  can  be  made  to  the  operational  version  of  the  PPSN 
model  that  will  enhance  the  product’s  output? 

2.  Which  PPSN  input  parameters  can  be  adjusted  to  decrease  computing 
memory  requirements  and  run  time? 

3.  Which  PPSN  input  parameters  interact  most  strongly  with  METOC 
conditions? 

4.  Which  PPSN  input  parameters  are  the  most  influential  in  determining  the 
distribution  of  relative  risk  of  pirate  presence  in  the  PPSN? 


22  Effective  March  1,  201 1,  the  name  of  the  next-generation  Piracy  Performance  Surface  model, 
PPSN,  was  changed  to  the  Pirate  Attack  Risk  Surface  model,  PARS. 
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To  answer  Questions  2,  3,  and  4  above,  a  systematic  approach,  not  a  trial-by-error 
approach,  to  experimental  designs  is  used.  The  DOE  use  a  robust  design  and  use  a 
realistic  model  of  PAG  CONOPS  that  conveys  operators'  knowledge  about  current  pirate 
behavior  as  seen  in  the  SBR  and  GOA. 

The  memory  requirement  findings  in  this  thesis  identify  the  three  key  input 
parameters  that  impact  computer  memory  requirement,  which  is  also  highly  correlated 
with  runtime.  These  variables  are  the  PAG's  mission  length,  the  number  of  PAG 
replications  in  the  simulation,  and  the  number  of  PAGs.  The  analysis  by  the  author 
concludes  that  the  values  inputted  for  each  of  these  variables  should  be  considered  with 
the  utmost  care  as  their  associated  values  greatly  impact  the  forecast  of  relative  pirate 
presence  probability.  For  the  PAG's  mission  length,  the  value  associated  with  this  input 
variable  should  be  based  on  current  PAG  CONOPS  and  not  based  on  computing 
constraints;  for  example,  if  a  PAG  has  the  ability  to  be  at  sea  for  up  to  four  weeks,  then 
the  mission  length  should  be  set  accordingly.  Setting  the  input  value  to  anything  less  will 
sacrifice  validity  in  the  PPSN  output.  It  is  recommended  that  the  number  of  PAG 
replications  be  controlled  by  the  modelers  as  this  input  variable  is  not  indicative  of 
gathered  INTEL  or  METOC,  but  rather,  specific  to  the  PPSN  code.  The  last  variable,  the 
number  of  PAGs,  should  be  modeled  based  on  INTEL,  but  in  the  event  that  insufficient 
INTEL  is  available  then  the  operator  should  input  postulated  PAGs  into  locations  of 
typically  high  pirate  activity. 

The  search  pattern  the  PAG  uses  when  attempting  to  locate  a  target  vessel  of 
opportunity  is  the  most  significant  factor  in  the  PPSN  model.  This  variable  was 
introduced  into  the  PPSN  code  because  the  author  gained  intelligence  on  this  variable  and 
informed  the  developer  of  its  significance.  Operators  and  military  members  combating 
piracy  should  attempt  to  acquire  INTEL  on  the  specific  search  pattern  that  PAGs  use  in 
the  SBR  and  GOA. 

Another  significant  variable  that  is  driving  the  simulation  is  the  factor  by  which 
the  wind  impacts  the  PAG's  movement.  The  author  recommends  that  this  variable  be 
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controlled  in  the  model  by  METOC  and  piracy  subject  matter  experts  in  the  SBR  and 
GOA  as  they  have  a  greater  knowledge  in  this  area.  The  value  given  to  this  factor  does 
impact  the  output. 

An  interesting  input  variable  in  this  thesis  research  is  the  probability  that  PAGs 
have  METOC  information  prior  to  leaving  their  base  location.  In  the  linear  regression 
model,  this  variable  is  significant  and  shows  a  strong  interaction  with  the  PAGs'  search 
patterns  and  the  PAGs'  mission  lengths  when  analyzing  the  sensitivity  of  the  design 
points  to  a  change  in  METOC  across  the  forecast  lead  times.  Proper  INTEL  should  be 
gathered  on  the  specific  sophistication  of  PAG  camp  sites.  The  value  given  to  this 
variable  is  highly  influential  and  interacts  with  the  most  important  variable, 
search  _  pattern . 

The  results  of  this  thesis  did  not  come  at  an  easy  cost.  The  code  was  broken  in 
different  ways,  resulting  in  modifications  by  the  developer  to  make  PPSN  better  and 
more  robust,  which  delayed  the  completion  of  experiments  for  several  months.  The 
software  required  to  run  the  PPSN  model  was  not  available  on  the  Hamming  shared 
cluster  at  the  Naval  Postgraduate  School,  thus,  this  thesis  required  the  use  of  high 
performance  computing  power  using  the  hawk  from  Wright-Patterson  Air  Force  Base  in 
Ohio.  During  the  simulation  runs  of  this  thesis,  some  design  points  would  require  longer 
than  60  hours  of  computing  time,  and  there  were  even  times  when  this  runtime  was 
exceeded,  which  led  to  a  failed  design  point  run.  Computer  memory  calculations  show 
that  design  points  that  were  typically  below  3.5  GB  ran  to  completion  in  30  hours.  To 
ensure  that  all  the  design  points  ran  to  completion,  this  thesis  required  a  reduction  in  the 
PAG  mission  lengths  and  the  number  of  PAG  replications. 

C.  RECOMMENDATIONS 

The  following  recommendations  from  this  thesis  research  are  as  follows: 

1.  The  PAGs’  mission  lengths  and  the  number  of  PAGs  should  be  given 
realistic  values  despite  their  impact  on  computing  memory  requirements  and 
runtime.  Altering  these  values  to  unrealistically  low  values  will  have  a 
detrimental  impact  on  the  model. 
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2.  The  PAGs'  search  patterns  are  highly  influential  and  interact  most  strongly 
with  METOC  conditions,  so  INTEL  should  focus  on  accurately 
understanding  PAGs'  search  strategies. 

3.  Better  INTEL  on  the  number  of  PAGs  needs  to  be  provided  to  the  modelers. 
This  information  will  result  in  improved  forecasts  of  relative  pirate  presence 
probability. 

D.  THESIS  IMPACT 

The  author's  experience  tour  resulted  in  a  complete  overhaul  of  the  PPSN  code  by 
the  developer.  The  author's  sponsor,  Bill  Lingsch,  who  is  head  of  the  INTEL  Department 
at  CNMOC,  stated, 

Piracy  in  the  Somalia  over  the  last  two  years  has  dramatically  increased 
impacting  worldwide  commercial  shipping.  As  good  practices  are 
implemented  by  mariners  to  minimize  the  piracy  threat  the  pirates  tactics, 
techniques,  and  procedures  (TTPs)  are  evolving.  In  the  past  there  has 
been  anecdotal  evidence  of  how  and  when  pirates  would  attack.  The 
thesis  work  done  under  [the  author]  applies  the  much  needed  scientific 
rigor  to  understand  what  variables  really  matter  in  attempting  to  predict 
where  and  when  pirates  are  most  likely  to  attack.  These  results  are  huge 
as  Navy  moves  forward  in  building  a  forecast  model  for  areas  of  highest 
probability  of  attacks.  Task  Force  Commanders  can  then  use  this  product 
to  determine  where  to  allocate  limited  assets  in  a  vast  area  to  deter  and/or 
interdict  pirates.  [The  author's]  thesis  work  provides  the  groundwork  for 
both  the  operational  product  and  where  to  focus  follow  on  research.  Her 
engagement  throughout  the  operational  and  scientific  community  to 
address  piracy  has  truly  accelerated  support  to  the  operational  Navy  in 
addressing  this  growing  problem,  (personal  communication,  March  4, 

2011) 

The  after  action  report  provided  by  the  author  resulted  in  changes  made  to  the 
existing  PPSN  model.  While  this  research  was  being  conducted  the  PPSN  code  was 
being  reworked  and  altered.  This  created  an  intense  working  environment  and  a 
challenge  to  the  forward  progress  necessary  for  this  research. 

After  a  year  of  dedicated  hard  work  on  helping  to  provide  the  military  leaders  and 
operators  with  a  piracy  tool,  the  author  is  receiving  a  master's  degree  from  the  Naval 
Postgraduate  School  in  Operations  Research  a  week  after  the  PPSN  model  becomes 
operational.  This  is  a  landmark  in  the  history  of  military  analysis  as  it  is  the  first 
operational  prediction  product  that  couple  METOC  and  INTEL. 
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E.  FUTURE  RESEARCH 

Since  this  thesis  was  the  first  systematic  exploration  of  the  PPSN  simulation,  an 
abundant  amount  of  follow-on  research  exists.  Some  of  the  future  research  relating  to 
piracy  can  extend  to  the  following  areas: 

1 .  Differentiating  between  neutral  vessels  and  PAGs. 

2.  Providing  an  operational  risk  management  matrix  for  military  assets. 

3.  Guidance  for  the  proper  allocation  of  resources  (ships,  air,  unmanned 
systems)  to  cover  a  large  body  of  water. 

4.  Extension  to  the  issues  surrounding  drug  trafficking  on  the  high  seas. 

5.  Using  PPSN  to  reduce  uncertainty  about  the  location  of  camp  sites  once  a 
vessel  has  been  pirated. 

6.  Game  theory  approach  to  optimize  the  PAG's  best  course  of  action  and 
responses. 

7.  Finding  the  means  to  validate  the  model  against  historical  data. 
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APPENDIX  A.  AUTHOR’S  OR  EXPERIENCE  TOUR 


The  Naval  Postgraduate  School  operations  research  experience  tour  occurs  during 
the  last  six  weeks  of  the  5th  quarter.  During  this  time,  the  operations  research  student 
takes  time  out  of  the  rigors  of  the  previous  four  quarters  to  gain  infonnation  in  the  form 
of  intelligence  but  more  commonly,  data,  to  be  used  for  their  thesis  research.  The  author 
of  this  thesis  was  afforded  a  very  unique  experience  tour.  The  author  traveled  to  Stennis 
Space  Center  in  Mississippi;  Paris,  France;  Stuttgart,  Germany;  Bahrain  and  finished 
back  in  Monterey. 

The  first  stop  in  the  author's  experience  tour  was  to  Stennis  Space  Center  along 
the  Mississippi  Gulf  Coast  where  the  author  briefed  the  sponsor,  the  Naval  Meteorology 
and  Oceanography  Command.  During  this  time,  the  author  was  provided  invaluable 
information  on  the  PPS  model  and  its  background. 

Paris,  France,  was  the  next  stop  in  the  author's  experience  tour.  The  author 
presented  and  participated  in  the  NATO  Modeling  and  Simulation  Group  (MSG-088) 
conference.  The  lead  U.S.  Chainnan  for  this  group  is  Dr.  Gary  Home23,  Naval 
Postgraduate  School  Operations  Research  professor.  Dr.  G.  Horne  stated, 

The  context  of  the  Task  Group  is  to  work  on  NATO  related  issues.  The 
Task  Group  will  look  to  use  Data  Farming  in  the  6  realms:  High 
Performance  Computing,  Visualization  of  simulation  data  output,  Rapid 
prototyping  of  scenarios,  Design  of  experiments,  Collaborative  processes, 
and  Model  Development.  Each  of  the  members  that  have  participated  in 
the  MSG  will  be  a  subgroup  to  help  solve  the  designated  problem, 
(personal  communication,  May  26,  2010) 

At  the  MSG-088  conference,  the  author  did  not  gather  information  on  piracy,  but 
was  able  to  gain  information  on  different  techniques  used  by  other  countries  to  solve 
sophisticated  problems. 

The  third  stop  in  the  author's  operations  research  experience  tour  was  to  the 
United  States  African  Command  headquarters  in  Stuttgart,  Germany.  This  visit  was  vital 

23  Dr.  G.  Home  is  also  a  professor  in  the  Operations  Research  Department  at  the  Naval  Postgraduate 
School  in  Monterey,  CA. 
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for  the  sharing  of  information  on  the  current  PPS  model  and  the  future  PPSN  model, 
along  with  describing  how  the  author's  thesis  research  would  impact  and  enhance  the 
development  of  the  PPSN  model.  At  the  time  of  the  author's  experience  tour,  AFRICOM 
had  little  to  no  partnership  with  Naval  Postgraduate  School.  As  a  result  of  the  author's 
experience  tour,  a  new  relationship  was  formed  between  this  command  and  the  Naval 
Postgraduate  School,  specifically  with  the  Operations  Research  department. 

After  the  author's  presentation,  Lieutenant  Commander  Netherlands  Navy  (OF  3) 
J.  Bertelink24  provided  positive  feedback  and  recognized  the  possible  benefits  of  using 
the  PPSN  model  for  anti-piracy  operations  conducted  by  the  Netherlands  Navy. 
Lieutenant  Commander  Bertelink  (personal  communication,  September  7,  2010)  stated, 

One  of  my  duties  is  to  organize  and  (if  possible)  improve  the  METOC 
support  to  the  NLD  navy  units.  As  you  probably  know  the  NLD  navy 
plays  an  active  roll  (siz)  in  the  NATO  Operation  Ocean  Shield  and  the  EU 
Operation  Atalanta,  both  anti  piracy  operations  in  the  Somalia  basin. 
Therefore  your  presentation  about  the  PPS-next  model  looked  very 
interesting  to  me  and  I  would  like  to  ask  you  some  questions  about  it. 

The  final  stop  of  the  author's  experience  tour  was  to  the  Combined  Maritime 
Forces  (CMF)  headquarters  in  Bahrain.  The  briefing  room  reserved  for  the  author  was 
filled  with  a  multitude  of  high-ranking  NATO  and  EU  military  members  who  were 
concerned  with  the  increase  in  pirate  activity  within  their  area  of  concern,  the  SBR  and 
GOA.  They  were  familiar  with  the  PPS  model,  but  skeptical  and  yet  intrigued  by  the 
PPSN  model  that  would  supersede  PPS.  The  presentation  was  filled  with  questions  that 
helped  the  author  understand  what  tools  the  operators  were  requesting.  During  the 
remainder  of  the  author's  time  with  the  CMF  Assessment  Team,  the  author  was  able  to 
gain  important  intelligence  on  current  military  operations  and  pirate  definitions,  as  well 
as  CONOPS  that  occur  within  the  SBR  and  GOA. 

The  author,  now  equipped  with  a  wealth  of  knowledge  from  the  previous  stops  of 
the  experience  tour,  returned  to  the  Naval  Postgraduate  School  in  Monterey  for  the  final 
stop.  The  author  briefed  the  findings  of  the  experience  tour  to  the  developer,  modelers, 

24  Lieutenant  Commander  Netherlands  Navy  J.  Bertelink  is  head  of  the  Royal  Netherlands  Navy 
Command,  Hydrographic  Office. 
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and  sponsor,  which  led  to  the  redevelopment  of  the  PPSN  model.  This  version  of  the 
PPSN  model  written  in  Python  (x,y)  will  be  the  operational  model  used  in  combating 
pirate  activity  in  the  SBR  and  GOA. 
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APPENDIX  B.  MEMORY  REQUIREMENT  DOE 


Appendix  B  shows  the  variables  values  according  to  each  design  point  and  their 
respective  base  to  waypoint  mapping  input  values. 


t_mission 

varies 

according  to 

design  point 

npi 

varies 

according  to 

design  point 

numtracks 

varies 

according  to 

design  point 

nbase 

4 

See  below 

skiff  speed 

12 

See  below 

drift  curr 

1 

drift_wind 

0.3 

wind  thresh  (mean) 

13.0 

See  below 

wave_thresh  (mean) 

4.8 

iweather  (mean) 

0.5 

See  below 

search  pattern  (categorical) 

0.1, 2, 3 

Bases: 

Ion 

lat 

Ion  size  lat 

size  weight 

50.10528 

6.316111 

0.01 

0.01 

1 

46.92722 

2.361111 

0.01 

0.01 

1 

58.41361 

4.258333 

2 

2 

1 

46.29083 

-5.00556 

2 

2 

1 

Waypoints  Ion 

lat  lonsize 

latsize 

weight 

55.14861 

2.415556 

1 

1 

1 

43.40861 

-2.53222 

1 

1 

1 

58.41361 

4.258333 

2 

2 

1 

46.29083 

-5.00556 

2 

2 

1 

iikiff  Speed 


mean 

10 

10 

12 

10 


iweather 


0 

1 

0 

1 


METOC  Thresholds 


wind  wave 

15  4 

10  5 

12  6 

15  4 


search 


ittern 

0 

1 

2 

3 
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DP 


npi 

numtracks 

t  mission 

1 

4 

128 

36 

2 

4 

256 

36 

3 

4 

384 

36 

4 

4 

512 

36 

5 

4 

640 

36 

6 

4 

768 

36 

7 

4 

896 

36 

8 

4 

1024 

36 

9 

4 

1152 

36 

10 

4 

1280 

36 

11 

4 

1408 

36 

12 
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1536 

36 

13 
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1664 

36 

14 

4 
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36 

15 

4 

1920 

36 

16 

4 

2048 

36 

17 

4 

128 

24 

18 

4 

128 

30 

19 

4 

128 

42 

20 

4 

128 

48 

21 

4 

128 

60 

22 

4 

128 

72 

23 

4 

128 

84 

24 

4 

128 

96 

25 

4 

128 

108 

26 

4 

128 

120 

27 

4 

128 

144 

28 

4 

128 

168 

29 

4 

128 
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30 

4 

128 

216 

31 

4 

128 

240 

32 

4 

128 

264 

33 

4 

128 

288 

34 

4 

128 

312 

35 

4 

128 

336 

36 

8 

128 

36 

37 

12 

128 

36 

38 

16 

128 

36 

39 

20 

128 

36 

40 

24 

128 

36 

41 

28 

128 

36 

42 

32 

128 

36 

43 

36 

128 

36 

44 

40 

128 

36 

45 

44 

128 

36 

46 

48 

128 

36 
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APPENDIX  C 


SIGNIFICANT  FACTORS  DOE 


design 

point 

t  mission 

skiff  speed 

drift  curr 

drift  wind 

wind 

threshold 

wave 

threshold 

iweather 

search 

pattern 

npi 

numtracks 

1 

240 

18.13 

0.59 

0.48 

26.25 

7.13 

0.47 

3 

24 

634 

2 

224 

8.75 

0.69 

0.55 

21.88 

7.5 

0.31 

3 

19 

859 

3 

219 

32.19 

0.58 

0.46 

25.94 

7.31 

0.03 

1 

29 

438 

4 

167 

35.00 

0.70 

0.56 

21.56 

7.88 

0.06 

1 

12 

297 

5 

230 

19.06 

0.61 

0.49 

27.19 

5.44 

0.56 

0 

13 

719 

6 

235 

14.38 

0.64 

0.51 

22.19 

3.94 

0.88 

0 

21 

775 

7 

188 

34.06 

0.63 

0.50 

26.56 

5.25 

0.91 

3 

10 

353 

8 

161 

33.13 

0.67 

0.54 

22.5 

4.31 

1 

2 

29 

269 

9 

182 

11.56 

0.77 

0.61 

23.13 

3 

0.19 

2 

22 

156 

10 

198 

13.44 

0.84 

0.68 

25.31 

3.56 

0.38 

3 

14 

128 

11 

193 

27.50 

0.98 

0.79 

20.63 

3.75 

0.16 

1 

23 

691 

12 

203 

24.69 

0.97 

0.78 

29.69 

5.81 

0.41 

1 

13 

606 

13 

172 

10.63 

0.78 

0.62 

21.25 

8.81 

0.78 

1 

16 

100 

14 

214 

16.25 

0.94 

0.75 

25.63 

8.63 

0.72 

1 

26 

213 

15 

177 

30.31 

0.95 

0.76 

20 

6.94 

0.75 

2 

17 

916 

16 

209 

22.81 

1.00 

0.80 

29.06 

6.38 

0.66 

2 

25 

578 

17 

156 

20.00 

0.75 

0.60 

25 

6 

0.5 

2 

20 

550 

18 

72 

21.88 

0.91 

0.72 

23.75 

4.88 

0.53 

0 

16 

466 

19 

88 

31.25 

0.81 

0.65 

28.13 

4.5 

0.69 

0 

21 

241 

20 

93 

7.81 

0.92 

0.74 

24.06 

4.69 

0.97 

2 

11 

663 

21 

146 

5.00 

0.80 

0.64 

28.44 

4.13 

0.94 

2 

28 

803 

22 

83 

20.94 

0.89 

0.71 

22.81 

6.56 

0.44 

3 

28 

381 

23 

77 

25.63 

0.86 

0.69 

27.81 

8.06 

0.13 

3 

19 

325 

24 

125 

5.94 

0.88 

0.70 

23.44 

6.75 

0.09 

0 

30 

747 

25 

151 

6.88 

0.83 

0.66 

27.5 

7.69 

0 

1 

11 

831 

26 

130 

28.44 

0.73 

0.59 

26.88 

9 

0.81 

1 

18 

944 

27 

114 

26.56 

0.66 

0.52 

24.69 

8.44 

0.63 

0 

26 

972 

28 

119 

12.50 

0.52 

0.41 

29.38 

8.25 

0.84 

2 

18 

409 

29 

109 

15.31 

0.53 

0.42 

20.31 

6.19 

0.59 

2 

27 

494 

30 

140 

29.38 

0.72 

0.58 

28.75 

3.19 

0.22 

2 

24 

1000 

31 

98 

23.75 

0.56 

0.45 

24.38 

3.38 

0.28 

2 

14 

888 

32 

135 

9.69 

0.55 

0.44 

30 

5.06 

0.25 

1 

23 

184 

33 

104 

17.19 

0.50 

0.40 

20.94 

5.63 

0.34 

1 

15 

522 
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